humanoid4rent
Prihlásiť saZískať cenové ponuky

World models for robots, explained for non-engineers

World models let a robot's software imagine what happens next. Here is how NVIDIA, Meta, 1X and Google DeepMind use them, and how much has been shown outside the lab.

Autor: redakcia humanoid4rent · zverejnené 17. novembra 2026

Táto stránka zatiaľ nie je preložená
Čítate anglický text, pretože verzia v jazyku Slovenčina zatiaľ neexistuje. Nič nebolo vynechané: toto je celá stránka v jazyku, v ktorom bola napísaná.

A world model is an AI system that predicts how a scene will change, often as video, when something acts in it. Robotics teams use world models in three ways: to generate extra training data, to test a robot's skills in a virtual setting, and to let a robot plan by imagining outcomes first. Most results so far come from research papers and lab demonstrations, not from robots you can hire.

What a world model actually is

Think of it as a learned simulator. A traditional physics simulator needs engineers to build 3D models of every object and to set rules for friction, mass and contact. A world model skips most of that. It is trained on large amounts of video and learns, statistically, what usually happens next: a cup tipped over spills, a door pushed swings open, a cloth folded keeps its new shape.

The important extra ingredient for robots is the action. A general video generator turns a text prompt into a clip. A robot world model takes the current camera view plus a proposed movement, such as "move the right gripper 10 cm forward and close it", and predicts the camera view that would follow. That turns a video generator into something a robot's software can query: "if I do this, what happens?"

This is a different idea from the vision-language-action models that directly output motor commands. A VLA decides what to do; a world model predicts the consequences. In practice, the two are increasingly combined.

Why video pretraining matters

The main bottleneck in robot learning is data. Collecting demonstrations usually means a person teleoperating a robot through a task hundreds of times, which is slow and expensive. Video of the physical world, by contrast, exists in enormous quantities.

The bet behind video pretraining is that a model which has watched a very large amount of footage has already learned a good deal about objects, motion and cause and effect. Only a small amount of robot-specific data should then be needed to adapt it to one particular machine. Several published projects test exactly this bet.

Example 1: NVIDIA Cosmos, a shared base model

NVIDIA announced Cosmos at CES on 6 January 2025, describing it as a platform of "world foundation models" for robots and autonomous vehicles. In its launch press release, NVIDIA said its data pipeline could process 20 million hours of video in 14 days on its Blackwell hardware, and listed robotics companies including 1X, Agility, Figure AI, Fourier and Neura Robotics among early adopters. Being listed as an adopter does not, on its own, tell you how a company uses the models.

The accompanying technical report on arXiv describes the platform's parts: a video curation pipeline, pre-trained world models, video tokenisers that compress footage into a form the model can handle, and examples of post-training for specific uses. The weights are released openly under a permissive licence, so developers can fine-tune them on their own robot footage. Cosmos sits alongside NVIDIA's robot model work covered in our article on Isaac GR00T and Jetson Thor.

Example 2: DreamGen, generating training data for a humanoid

The clearest published example of a world model producing robot training data is DreamGen, an NVIDIA research project. According to the DreamGen project page, the method works in four steps:

  1. Fine-tune an existing video world model on footage of the target robot so it learns how that body moves.
  2. Give the model a starting image and a written instruction, and let it generate a video of the robot doing the task.
  3. Infer the motor commands that would have produced that video, using a separate model trained to work backwards from motion to actions.
  4. Train the robot's control policy on these generated videos and inferred actions, which the authors call "neural trajectories".

The team tested several video models, including Cosmos and WAN 2.1, on a Fourier GR1 humanoid, a Franka arm and the low-cost SO-100 arm. The authors report that the humanoid learned 22 new behaviours in 10 new environments while the real teleoperation data came from only a single pick-and-place task in one environment. They also report that policy performance rose steadily as more synthetic trajectories were added.

That is a significant research result, but it is a lab result. The tasks are short manipulation actions on a table, measured by the authors themselves, and the authors do not claim the method is in use on commercial robots.

Example 3: Meta V-JEPA 2, planning by imagining

Meta took a different route with V-JEPA 2, published on 11 June 2025. Instead of predicting every pixel of the next frame, it predicts an abstract summary of what the scene will look like, which is cheaper to compute. According to the V-JEPA 2 paper, the model was pretrained on more than a million hours of internet video and images, then adapted with fewer than 62 hours of robot video into an action-conditioned version for planning.

Here the world model is used for planning rather than data generation. The robot is shown a picture of the goal, imagines the result of several candidate movements, and picks the one whose imagined outcome is closest to the goal. Meta says it ran this on Franka arms in two labs without collecting data in those environments. In its announcement blog post, Meta reports success rates of 65% to 80% for picking and placing new objects in unseen settings. Meta also notes limits: the model plans at only one time scale, so long multi-step jobs remain hard, and the robot dataset is small.

A success rate of 65–80% is a real step for a robot working without task-specific training. It also means one attempt in five or more fails, which is far below what a live audience or a production line would accept.

Example 4: 1X, a world model as a test bench

The home-robot maker 1X uses world models mainly to evaluate robot software. Its September 2024 post on the 1X World Model explains the problem: when engineers change a robot's control model, it is hard to know whether it really got better, because real homes and offices never stay the same. A world model trained on thousands of hours of the company's own robot footage lets them replay the same starting scene with different actions and compare outcomes.

1X is unusually frank about the weaknesses. The post says objects can change shape or colour or vanish, physics is not applied consistently, and objects sometimes appear to ignore gravity. The company also released more than 100 hours of data and launched a public challenge for researchers.

In June 2026, 1X announced a dedicated World Model Lab, saying it pretrains on web video, first-person human video, simulation, teleoperated robot data and data from its NEO robots in use. The company says this work has let NEO carry out some unseen tasks zero-shot, meaning without task-specific training. The announcement gives no independent measurement, and NEO's real-world operation also relies on remote human help, as covered in our article on 1X NEO and teleoperation.

Example 5: Genie 3, worlds for agents to train in

Google DeepMind's Genie 3, announced on 5 August 2025, generates interactive 3D environments from a text prompt at 720p and 24 frames per second, and keeps them consistent for a few minutes. DeepMind showed its SIMA agent, which plays in virtual 3D worlds, pursuing goals inside generated scenes.

It is worth being precise: Genie 3 was presented as a step towards training embodied agents, but the published demonstrations involve virtual agents, not physical robots. DeepMind lists limits including a narrow range of actions, difficulty with several independent agents, and interaction lasting minutes rather than hours. It was released only as a limited research preview.

Papers versus products

Across these releases, a pattern is clear:

  • Demonstrated in papers: generating synthetic robot videos that improve learned skills (DreamGen), and planning short pick-and-place tasks by imagining outcomes (V-JEPA 2), both on lab robots with author-reported results.
  • Used inside companies: evaluating new robot software against a learned simulator (1X), and fine-tuning open base models such as Cosmos on company footage. Details of how much this affects shipped robots are mostly not published.
  • Not yet shown: a commercial humanoid whose everyday behaviour in public venues is planned by a world model and verified independently.

World models also share a basic weakness: they predict what looks plausible, not what is physically guaranteed. This is why they complement, rather than replace, traditional physics simulation, described in our explainer on learning in simulation, and real-world testing.

What this means for event organisers

  • Expect faster skill development, not instant autonomy. World models may shorten the time suppliers need to teach a robot new moves, but robots at events today still mostly run scripted routines or are operated by people; see teleoperated or autonomous robots.
  • Read success rates carefully. A lab figure of 65–80% on table-top tasks says little about reliability in a crowded hall.
  • Ask what is live and what is generated. As video generation improves, some promotional clips may be synthetic or heavily edited. It is reasonable to ask a supplier for unedited footage of the exact robot and routine you are booking.
  • Safety limits do not change. Whatever software trained the robot, the physical safety measures, operator presence and site rules for the event still apply.

Frequently asked questions

Is a world model the same as a physics simulator?

No. A physics simulator calculates motion from engineer-built 3D models and explicit rules for mass, friction and contact. A world model learns from video what usually happens next and generates a prediction. It needs far less manual setup and can cover messy scenes, but it can produce physically impossible results, such as objects vanishing, so teams use both and still test on real robots.

Do humanoid robots at events use world models today?

There is no public evidence that robots hired for events plan their behaviour with world models. Event robots usually run pre-programmed routines, remote operation or a combination. World models are used mainly in research and inside robot companies to generate training data and evaluate software, so any effect on event robots is indirect, through how their makers develop new skills.

What does zero-shot mean in robot world model announcements?

Zero-shot means the robot performed a task or worked in an environment without being given training examples for that specific case. Meta and 1X both use the term for their results. It does not mean the robot succeeds every time: Meta reports success rates between 65% and 80% for its zero-shot pick-and-place tests in new settings.

Why do world models need so much video?

To predict what happens next, a model has to have seen many examples of objects moving, falling, opening and being handled. Internet video provides that variety cheaply. Meta's V-JEPA 2 was pretrained on more than a million hours of video and images, then adapted with fewer than 62 hours of robot footage, showing how video pretraining reduces the robot data needed.

How this article was made: written by the humanoid4rent editorial team with AI assistance; every factual statement was checked against the sources listed below on 2026-11-17. Spotted an error? Write to info@humanoid4rent.com.

Zdroje

Fakty v tomto článku boli overené podľa týchto stránok v deň jeho vzniku.

  1. Cosmos World Foundation Model Platform for Physical AI – arXiv
  2. NVIDIA Launches Cosmos World Foundation Model Platform to Accelerate Physical AI Development – NVIDIA Newsroom
  3. DreamGen - Unlocking Generalization in Robot Learning through Video World Models – NVIDIA Research
  4. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning – arXiv
  5. Introducing the V-JEPA 2 world model and new benchmarks for physical reasoning – Meta AI
  6. 1X World Model – 1X Technologies
  7. 1X Launches World Model Lab to Scale Humanoid Intelligence – 1X Technologies
  8. Genie 3: A new frontier for world models – Google DeepMind

Ďalšie z blogu

Chcete robota na svojom nasledujúcom podujatí?

Popíšte raz termín, miesto a to, čo má robot robiť. Dodávatelia odpovedia ponukami, ktoré môžete porovnať na rovnakom základe.

Odoslať dopyt

Pracujete v robotike? Píšte pre náš blog. →