Learning in simulation: how robots get new skills before they touch the floor

Sim-to-real training means a robot learns a skill inside a physics simulation, through a very large number of practice attempts, and the resulting control software is then loaded onto the physical machine. Developers work this way because simulated practice is fast, cheap and does not damage hardware. The difficulty is the reality gap: no simulator is perfect, so engineers deliberately vary it to make skills robust.

This explainer covers how the method works, the research behind it, how manufacturers ship learned skills to robots already in the field, and what the approach still cannot do.

Why robots practise in a simulator

Most learned robot skills today come from reinforcement learning. The robot, or rather a software model of it, tries an action, receives a score from a so-called reward function (for example, points for moving forward and penalties for falling), and gradually adjusts a neural network so that higher-scoring actions become more likely. Boston Dynamics summarises the appeal neatly in a 2024 post on reinforcement learning for Spot: engineers mainly need to be able to simulate the situations they care about and define what good performance looks like.

Doing this on real hardware would be slow and destructive. A legged robot learning by trial and error falls thousands of times, and every fall costs time, wear and repairs. In a simulator, many virtual copies of the robot can practise in parallel on graphics processors.

The speed-up is dramatic. In a 2021 paper, Nikita Rudin and colleagues reported training walking behaviour for the ANYmal quadruped in under four minutes for flat ground and about twenty minutes for uneven terrain, using a single workstation GPU, and then running the result on the real robot. Boston Dynamics makes a similar point about its humanoid: in a post on teaching Atlas a football trick, the company says parallel GPU simulation in the cloud gives Atlas the equivalent of a year of physical trial and error in 24 hours.

The reality gap

A simulator is a simplified model of the world. Friction, the way electric motors respond under load, the flex of a rubber foot, sensor noise, small communication delays and lighting are all approximated. A neural network trained in that model will happily exploit any quirk it finds. If simulated feet grip slightly better than real ones, the learned gait may rely on that grip and slip on a real floor.

The gap between how a skill performs in simulation and how it performs on hardware is what researchers call the reality gap, or sim-to-real gap. Much of the progress over the past decade has been about narrowing it, either by making simulators more faithful or by making the learned skill indifferent to the differences.

Domain randomisation: training for a world that is never quite the same

The most widely used technique is domain randomisation. Instead of trying to build one perfect simulator, engineers build a family of slightly different ones and train across all of them. The reasoning is that if the skill copes with a wide spread of variations, the real world looks to it like just one more variation.

An early demonstration came from a 2017 paper by Josh Tobin and colleagues. They randomised the textures and rendering of simulated scenes and trained an object detector only on those synthetic images. On real camera images, the authors report it located objects to within 1.5 cm, coped with distracting objects and partial occlusion, and was good enough to guide a robot grasping objects in clutter.

For movement, the same idea is applied to physics, often called dynamics randomisation: body mass, friction, motor strength and reaction delays are varied from one practice episode to the next. Developers often add random pushes during training so that the robot learns to recover.

A refinement is to let the system widen the variations itself. OpenAI's 2019 Rubik's cube work introduced automatic domain randomisation, which steadily increases the range of randomised conditions as the policy improves. The team used it to train a robotic hand, entirely in simulation, to manipulate and solve a Rubik's cube on real hardware.

What the research has shown

Two papers in Science Robotics are often cited as milestones for legged robots.

In 2019, Jemin Hwangbo and colleagues trained locomotion policies for ANYmal in simulation and transferred them to the real quadruped. According to the paper, the robot followed speed commands precisely and energy-efficiently, ran faster than with its previous controller and recovered from falls, including from awkward positions.

In 2024, a Google DeepMind team led by Tuomas Haarnoja published results for a miniature humanoid playing football. The robot, with 20 actuated joints, was trained in simulation for a simplified one-against-one game and transferred to the real robot without further training, which researchers call zero-shot transfer. Compared with a scripted baseline controller, the authors report that it walked 181% faster, turned 302% faster, took 63% less time to get up after a fall and kicked the ball 34% faster. They name three ingredients as essential: a high control frequency, targeted dynamics randomisation and perturbations during training.

These are research results on specific robots in controlled settings. The DeepMind robot was small and played on a prepared pitch; the numbers are relative to a scripted controller, not to a human or to other humanoids.

Why new moves can arrive via a software update

Once training is finished, a learned skill is essentially a file: the weights of a neural network that turns sensor readings into motor commands many times per second. The motors, sensors and frame stay the same. That is why a manufacturer can add or improve a behaviour without touching the hardware.

Boston Dynamics described exactly this route for Spot. Its locomotion policy was trained over more than a million simulations, checked against simulated benchmarks of increasing difficulty, and tested on an internal fleet that logged more than 2,000 hours of testing each week before release. Existing customers received the improvement with the Spot 4.0 software update, and the company says it keeps refining the policy with fleet data.

For Atlas, Boston Dynamics describes a three-step pipeline for its football trick, created for Hyundai's campaign ahead of the 2026 FIFA World Cup: a player's movement was recorded with optical motion capture, adapted to the robot's different body proportions, and then refined with reinforcement learning in simulation. The company says that for almost all of Atlas's skills, the learned policy worked first time on the robot. Note the word almost.

The tooling is also becoming more standardised. NVIDIA's Isaac Lab paper, posted in November 2025 with more than 100 contributors, presents the open framework as the successor to Isaac Gym. It bundles GPU-accelerated physics and rendering, actuator models, sensor simulation and built-in domain randomisation, and the authors show uses ranging from whole-body control to dexterous manipulation. Our article on NVIDIA Isaac GR00T and Jetson Thor covers the related software stack.

Limits worth knowing

Sim-to-real training is powerful but it does not make robots general-purpose.

• A skill is only as robust as its training. A policy has experienced only the variations its developers simulated. A floor, slope, cable or crowd that was never modelled is outside that experience.
• Transfer is not guaranteed. Even manufacturers that use the method extensively describe a validation stage on real hardware, and some skills need further tuning.
• Contact with people and soft objects is hard to simulate. Walking and balance are comparatively well understood; handling varied objects or interacting physically with people remains much harder to model.
• Learned controllers are harder to inspect. A neural network does not come with a readable list of rules, which is why testing and safety limits around it matter.
• Showcase moves differ from routine work. A trained kick, flip or dance demonstrates what a controller can do under the conditions it was trained and tested for, not that it will perform equally well anywhere.

For the mechanics of staying upright, see our explainer on how humanoid robots keep their balance.

What this means for event organisers

• New moves are often a software question, not a hardware one. If you want a specific movement, ask whether it already exists in the robot's software or would need new development, and allow for the lead times described in our guide to custom moves and choreography.
• Ask where a move has been tested. A skill proven in a lab or on a film set may behave differently on your venue's floor. Include the surface, space and audience distance in your brief to suppliers.
• Check the software version before the day. Manufacturer updates can change how a robot walks or moves. Suppliers should test the exact version they will run at your event and avoid updating at the last minute.
• Plan for the gap. Learned skills can still fail in unfamiliar conditions, so agree a fallback in advance, as outlined in what happens if the robot fails.

Frequently asked questions

What does sim-to-real mean in robotics?

Sim-to-real describes training a robot skill in a computer simulation and then transferring the resulting control software to a physical robot. The robot practises a task many times in a virtual world, guided by a reward score, and the finished neural network is loaded onto the real machine. The main challenge is the reality gap: differences between the simulated and real world that can make a skill behave differently on hardware.

What is domain randomisation?

Domain randomisation is a technique for making simulated training more robust. Instead of using one fixed simulator, developers vary its settings from episode to episode, such as textures and lighting for vision, or mass, friction, motor strength and delays for movement. A skill that copes with that spread of conditions is more likely to work in the real world, which then appears to the robot as just one more variation.

Can a robot learn a new dance or move through a software update?

Often, yes. A learned movement is stored as software, so a manufacturer or supplier can add or improve behaviours without changing the hardware. Boston Dynamics, for example, delivered a simulation-trained walking controller to existing Spot owners in a software update. New moves still need testing on real robots, and custom choreography for a particular event takes development and rehearsal time.

Does training in simulation make robots safe around the public?

No, not by itself. Simulation lets developers test many scenarios without risk, but a learned skill only reflects the conditions that were modelled. Crowds, unusual floors and physical contact with people are hard to simulate accurately. Safe operation at events still depends on the manufacturer's safety limits, trained operators, clear distances between robot and visitors, and a venue risk assessment.

How this article was made: written by the humanoid4rent editorial team with AI assistance; every factual statement was checked against the sources listed below on 2026-10-31. Spotted an error? Write to info@humanoid4rent.com.