Born to Learn

Changing the behaviour, keeping the model.

A body schema is an internal model of the body: where its parts are, how they move, and what they can reach. We studied how movement and self-touch help a simulated body learn that model.

We kept the body, model architecture, training objective and data budget fixed. We changed the behaviour used to collect the training data, then measured the resulting representations.

Two ways to explore

We used the MIMo infant body in MuJoCo, with an 88-dimensional proprioceptive and tactile state and an 88-dimensional muscle action space.

One approach trained five Soft Actor-Critic specialists, each targeting a body region: head, upper torso, lower torso, legs or arms. Switching between them gave us a mixture of experts (MoE). The other used a single skill-conditioned actor trained with a quality-diversity objective (QDAC), with continuous control over which regions it explored.

We collected 32 dataset sources across structured policies, random movement and scripted behaviour. We described them by self-touch frequency and diversity, use of both hands, movement coherence, vigour and skill-space coverage.

Each dataset trained a fresh body-schema model: a dual variational autoencoder over proprioception and touch, plus a residual model predicting state changes. The optimiser, training budget and seeds stayed the same.

What we measured

Reconstructing an input accurately doesn’t tell us whether a representation is useful for controlling a body. We used six probes, on both in-distribution data and a shared held-out set:

What we found

In these experiments, the structured exploration policies produced better body schemas than random movement and babbling. Every MoE schema ranked above every baseline in the composite evaluation.

Coordinated movement was more useful than vigorous movement. Movement coherence correlated positively with the probes: Spearman correlations were +0.75 for RSA-joint, +0.70 for MIG-joint and +0.63 for two-point discrimination. Pose vigour was the strongest negative predictor.

Accurate self-touch didn’t guarantee a better representation. QDAC was better at targeting body regions during policy training, but the MoE datasets produced stronger body schemas. A re-evaluation of the Yang & Jamone self-touch baseline placed it between babbling and the structured families.

Checkpoint choice mattered too. QDAC’s reward continued to improve after representation quality had peaked. The probes improved up to around one million training steps, plateaued near 1.5 million, and fell back towards baseline by three million. Selecting a checkpoint by reward alone missed that window.

Behaviour is part of the design.

We got different body schemas from the same model and training budget just by changing how the body explored. That makes behaviour worth designing and evaluating in its own right. These results come from a simulated body and this particular training setup; the next question is how well they transfer.