Whole-body control: how a humanoid coordinates its arms, legs and torso at once

Whole-body control is the software that decides, many times per second, what every joint of a humanoid should do so that walking, reaching and carrying happen together without the robot falling over. Older systems solved this with physics models and optimisation; newer ones mostly use neural networks trained in simulation, often by imitating recorded human movement. That imitation is why robot dance routines can look strikingly smooth.

Why moving the whole body is hard

A humanoid is not a set of independent limbs. When it raises an arm holding something heavy, its centre of mass shifts, and the legs must compensate immediately. When it takes a step, the swinging leg pushes the torso, which disturbs the hands. Every movement affects every other movement.

The only things that keep the robot upright are its feet pressing against the floor, and the forces they can apply are limited. Our separate explainer on how humanoid robots keep their balance covers that part. Whole-body control goes a step further: it is about doing useful things with the arms and torso while staying balanced, rather than treating balance and manipulation as separate jobs.

A modern humanoid can have dozens of motors. Boston Dynamics, for example, describes its Atlas as having 50 degrees of freedom (independently controllable joint movements) in its Large Behavior Models blog post. Coordinating that many joints by hand-written rules quickly becomes impractical.

The model-based approach: predict, then choose

The classical answer is model-based control. Engineers give the robot a mathematical description of its own body: the mass of each part, the length of each limb, how motors respond. A controller then uses this model to predict what will happen over the next fraction of a second and picks the joint commands that best achieve its goals.

A widely used form is model predictive control (MPC). It repeatedly plans a short way ahead, executes the first small piece of the plan, then re-plans with fresh sensor data. Goals are usually ranked: do not fall first, keep the feet where planned second, put the hand where requested third.

The strength of this approach is that it is understandable. Engineers can see why the robot did something and can set hard limits. The weakness is that the real world never quite matches the model: slippery floors, soft carpets, a box heavier than expected. Writing rules for every such case is slow work.

The learning approach: practise millions of times in simulation

Reinforcement learning (RL) takes a different route. Instead of writing the controller, engineers write a reward: points for moving forward, for keeping the torso upright, for following a command; penalties for falling or wasting energy. A neural network then controls a simulated robot and gradually learns, by trial and error, which actions earn the most reward.

Because simulation runs far faster than reality, the network can practise for the equivalent of years. Engineers vary friction, weights and terrain between runs so the policy does not overfit to one perfect virtual world. The trained network is then copied onto the real robot. Our article on learning in simulation explains this transfer step in more detail.

In practice the two approaches are often combined. Boston Dynamics has described adding a learned policy to Spot's walking while keeping its existing model-based controller, saying in a blog post on reinforcement learning for Spot that this lets the model-based part handle what it does well while learning focuses on what is hard to program. The company reports training the policy across more than a million simulations and testing it on an internal fleet before release to customers.

Motion imitation: learning from human movement

A pure reward for "walk forward without falling" tends to produce odd, unnatural gaits. Robots find strange shortcuts. A key idea that changed this is to reward the robot for copying a human.

The 2018 paper DeepMimic, trained simulated characters with a reward that combined two parts: how closely the character matched a reference motion clip, such as motion-capture data of a person, and how well it achieved a task goal. The result was physically simulated characters performing acrobatic skills while still reacting to pushes and changes in the environment.

Applying this to real humanoids adds a step called retargeting. Human motion-capture recordings are mapped onto the robot's different proportions and joint limits, because a robot cannot copy every human movement exactly.

The 2024 paper Expressive Whole-Body Control for Humanoid Robots (ExBody), made a practical compromise. The upper body imitates the reference motion, while the legs are only asked to follow a requested walking speed robustly. The authors report real-world walking in different styles, handshakes and dancing with people. The idea is simple: arms can afford to be expressive, but feet must prioritise not falling.

Why robot dance moves look so smooth

When a humanoid performs a polished dance at a gala or launch, it is usually not improvising. In the imitation-based methods described above, the controller is tracking a reference motion that was recorded or authored in advance, typically a human performance retargeted to the robot. Human movement is naturally fluid, and the reward explicitly pays the robot for matching it, so the result inherits that fluidity.

The neural network's job is to make that pre-planned motion physically possible on the real body: shifting weight, adjusting foot placement and absorbing small errors. That is why a routine can look effortless and yet be fragile in an unfamiliar spot, such as an uneven stage or a costume that changes the robot's weight. This is also why custom choreography needs lead time: each new routine has to be prepared, adapted to the robot and rehearsed.

Research is pushing beyond single clips. BeyondMimic, first posted on arXiv in August 2025, trains a motion-tracking controller for agile skills such as cartwheels, spin-kicks and sprinting, then uses a generative (diffusion) model to combine and adapt those skills for new tasks such as avoiding obstacles. The authors report transfer to real hardware; as with all research papers, those results come from controlled demonstrations rather than public events.

Closing the gap between simulation and reality

A motion that works in simulation can fail on hardware because motors, gearboxes and friction behave slightly differently. The 2025 paper ASAP, first posted in February 2025, tackles this on a Unitree G1. The team first trains motion-tracking policies in simulation from human motion data, then runs them on the real robot, records how reality differed, and trains a small correction model that is fed back into the simulator for further fine-tuning. The authors report that this made more agile motions possible than before.

From dancing to doing work

Imitating dance does not by itself make a robot useful. For tasks, researchers increasingly use humans as live pilots. OmniH2O, posted in June 2024, describes a single learned controller that follows a target body pose, which can come from a person wearing a VR headset, from a camera or from a higher-level AI model. The authors used the resulting teleoperated demonstrations to train autonomous skills and released a dataset of six everyday tasks. Our teleoperation explainer covers how such piloting works at events.

Boston Dynamics and Toyota Research Institute have taken a similar path with Atlas. In their Large Behavior Models post, they describe operators in VR headsets with foot and chest trackers demonstrating tasks, supported by Boston Dynamics' model predictive controller. A single language-conditioned policy then controls the whole robot, including stepping, squatting and the hands, from camera images, body sensors and text prompts, at 30 decisions per second. Demonstrated tasks included tying rope, flipping furniture and handling a tyre of about 22 pounds (10 kg). The team says much work remains.

IEEE Spectrum reported in September 2025 that Boston Dynamics is focusing this work on logistics tasks such as moving parts and totes rather than acrobatics, and plans to add reinforcement learning to improve robustness.

The limits that still apply

• Routines are rehearsed, not understood. A tracked dance follows a prepared reference; it does not respond to music or crowd mood unless someone builds that in.
• Carrying changes the physics. A load, a costume or a tray alters weight distribution, and controllers trained without it may behave differently.
• Learned controllers are harder to explain. When a neural policy misbehaves, engineers cannot always say why, which matters around the public.
• Research results are not product specifications. Paper demonstrations happen under controlled conditions with expert teams on hand.

What this means for event organisers

• Expect smooth choreography to be pre-built: ask the supplier which routines already exist on the robot and how long a new one would take.
• Tell the supplier about anything the robot will wear or carry, and about the floor surface, as both affect whole-body behaviour.
• Ask whether a segment will run autonomously or be piloted by an operator, and say so honestly to your audience if needed.
• Keep clear space around dynamic moves; fluid motion is not the same as safe contact. Our guide to robot safety at public events sets out practical distances and supervision.
• Treat research videos as direction of travel, not as what a hired robot will do on your stage next month.

Frequently asked questions

What does whole-body control mean for a humanoid robot?

It is the control software that coordinates all of a humanoid's joints at once, so that walking, balancing, reaching and carrying work together. Instead of moving the arms and legs separately, the controller accounts for how each movement shifts the robot's weight and adjusts the rest of the body. It can be built from physics models, from neural networks trained in simulation, or from a combination of both.

Why do humanoid robot dances look so smooth?

In many research methods, the robot is tracking a motion prepared in advance, often a recorded human performance adapted to the robot's body. The training rewards the robot for matching that reference closely, so it inherits the fluidity of human movement. The controller's job is to keep the motion physically possible and balanced. The routine is rehearsed rather than improvised, which is why new choreography needs preparation time.

Is reinforcement learning replacing traditional robot control?

Not entirely. Learned controllers are increasingly common, but companies often combine them with model-based methods. Boston Dynamics has described keeping its model-based controller for Spot while adding a learned policy for harder situations, and it used its model predictive controller to support teleoperated data collection for Atlas. Hybrid systems try to keep the predictability of models and the adaptability of learning.

Can a humanoid that dances well also carry objects at an event?

Not automatically. Dancing and carrying draw on different skills, and a load changes how the robot's weight is distributed. A controller trained for a dance may not have been trained with a tray or box in its hands. Ask the supplier specifically whether the robot has been tested carrying the kind of item you have in mind, and under what supervision.

How this article was made: written by the humanoid4rent editorial team with AI assistance; every factual statement was checked against the sources listed below on 2026-11-04. Spotted an error? Write to info@humanoid4rent.com.