Vision-language-action models: how RT-2, π0, Helix and Gemini Robotics control robots

A vision-language-action (VLA) model is an AI system that takes in camera images and a written or spoken instruction and outputs robot movements directly. Instead of an engineer scripting every motion, the model learns from internet data and recorded demonstrations. Google DeepMind, Physical Intelligence and Figure have all published such models since 2023, but none of them yet replaces a rehearsed event routine.

From scripts to learned behaviour

Most robots that appear at events today run on a mix of pre-programmed motion sequences and live remote control. A wave, a dance or a walk to a marked spot is designed in advance, tested and replayed. If something unexpected happens, a human operator takes over. Our guide to teleoperated and autonomous robots explains how this looks from the audience's side.

VLA models aim at something different. Rather than following a fixed list of joint positions, the robot looks at the scene, reads an instruction such as "put the red cup on the tray", and works out the movements itself. The same model is meant to handle many tasks and objects it was never explicitly programmed for.

This is why the term has become central in humanoid robotics announcements. It is also why claims about it need careful reading: a research demonstration in a lab kitchen is not the same as reliable behaviour in a crowded exhibition hall.

What the three words mean

The name describes the inputs and the output:

• Vision: the model receives images from the robot's cameras.
• Language: it receives an instruction in natural language, and it usually starts from a model already trained on large amounts of text and images from the internet.
• Action: instead of answering with text, it produces motor commands, such as where the arm or gripper should move next.

The key idea is reuse. A vision-language model already knows what a cup, a banana or a recycling bin looks like, and how people describe them. Researchers then train it further on recordings of robots doing tasks, often collected by humans steering the robot remotely. The model learns to connect what it sees and what it is told with what the robot should do.

RT-2: actions written as text

Google DeepMind introduced RT-2 on 28 July 2023 and described it as a vision-language-action model. According to the RT-2 announcement from Google DeepMind, the team adapted existing vision-language models and represented each robot action as a short string of numbers, handled like words in a sentence. The numbers encode things such as position changes, rotation and whether the gripper should open or close.

The robot data came from demonstrations recorded over 17 months with 13 robots in an office kitchen, combined with web-scale image and question-answering data. Google DeepMind reported that RT-2 succeeded in 62% of tested scenarios it had not seen in training, compared with 32% for its predecessor RT-1. It also showed simple reasoning, for example choosing an object that could serve as an improvised hammer.

RT-2 established the basic recipe, but it controlled a single-arm robot in a fairly controlled setting, and success in roughly six cases out of ten is far from what a live event needs.

π0 and π0.5: smoother, continuous motion

The start-up Physical Intelligence published π0 (pronounced "pi-zero") on 31 October 2024. Its π0 announcement describes a 3-billion-parameter vision-language model extended with a technique called flow matching, which produces continuous movement commands rather than text-like tokens. The company says π0 can output motor commands up to 50 times per second, which matters for delicate tasks such as folding laundry.

π0 was trained on data from eight different robot types, including single and two-armed setups and mobile platforms, plus open datasets. Demonstrated tasks included folding clothes, clearing a table and assembling a box. The company itself wrote that generalist robot policies are still in their infancy.

On 22 April 2025, Physical Intelligence followed with π0.5. The π0.5 announcement focuses on working in homes that were not part of the training data, with tasks such as tidying a kitchen or making a bed. The model first decides on a next step expressed in words, then generates the movements for it. Physical Intelligence states plainly that π0.5 is far from perfect and often makes mistakes, both in what it decides to do and in how it moves.

Helix: a fast and a slow system in one humanoid

Figure presented Helix on 20 February 2025 as a VLA model for its humanoid robots. As described in Figure's Helix post, it splits the work into two parts:

• System 2, a 7-billion-parameter vision-language model that interprets the scene and the instruction at 7 to 9 times per second.
• System 1, a much smaller 80-million-parameter model that turns that understanding into motor commands at 200 times per second across 35 degrees of freedom.

The idea is to think slowly about the goal while reacting quickly to what the hands and arms are doing. Figure says Helix runs on two low-power GPUs inside the robot, was trained on about 500 hours of teleoperated demonstrations, and uses one set of model weights for all its behaviours. Its demonstrations showed robots picking up household items they had not seen before and two robots coordinating on a shared task without being assigned fixed roles.

These are company demonstrations, not independent tests. For a look at how Figure's later hardware has been presented, see our article on what the Figure 03 footage actually shows.

Gemini Robotics: reasoning, then acting

Google DeepMind announced Gemini Robotics on 12 March 2025. The Gemini Robotics announcement describes it as a VLA model built on Gemini 2.0 that adds physical actions as a new type of output. A companion model, Gemini Robotics-ER, focuses on spatial understanding and is meant to work alongside robot control software written by developers. The models were shown on a two-armed research platform, ALOHA 2, on Franka arms and on Apptronik's Apollo humanoid.

Google DeepMind highlighted three goals: handling new situations, responding to conversational instructions and changes in the scene, and dexterous tasks such as folding origami. It also released a dataset called ASIMOV for evaluating whether robot behaviour is semantically safe, meaning whether an action makes sense and avoids harm in context.

Two updates followed. On 24 June 2025, the company presented Gemini Robotics On-Device, a version that runs on the robot without a network connection and can be adapted to new tasks with 50 to 100 demonstrations, according to Google DeepMind. On 25 September 2025, Gemini Robotics 1.5 added a step in which the model reasons in natural language before acting, and showed skills learned on one robot transferring to others. At that announcement, the ER 1.5 reasoning model was offered to developers through the Gemini API, while the action model itself remained limited to selected partners.

What VLA models still cannot do

Across all four developers, the published material points to the same limits:

• Reliability: reported success rates and the developers' own caveats show that these models still fail regularly, especially in unfamiliar settings.
• Scope: most demonstrations involve tabletop or household manipulation, not walking through crowds. Balance and locomotion are usually handled by separate controllers; see our explainer on how humanoid robots keep their balance.
• Availability: several of these models are research systems or are released only to partners, so a robot you can hire is unlikely to be running them.
• Verification: performance figures come from the companies themselves. Independent, comparable benchmarks for real-world use are still limited.

Safety is handled in layers. Google DeepMind, for example, describes combining the model's reasoning about safety with separate low-level safety systems. A learned model alone is not a safety guarantee, which is why physical limits, operators and site rules remain essential.

What changes for scripted event behaviour

For events, the practical shift is gradual. Choreography, stage entrances and timed speeches will continue to be programmed and rehearsed, because they must happen the same way every time. Our guide to custom programming and choreography covers the lead times this involves.

Where VLA models could matter first is in interactive moments: handing an item to a guest, sorting objects on a demo table, or responding to an unplanned request. Those are exactly the situations where today's models are least predictable, so any supplier offering them should be able to explain what is learned, what is scripted and who can intervene.

What this means for event organisers

• Treat VLA announcements as research progress, not as a description of what a rental robot will do at your event.
• Ask suppliers which parts of a performance are pre-programmed, which are autonomous and which are teleoperated, and put the answer in your brief to the supplier.
• For any autonomous interaction near guests, ask how the robot is stopped, what speed and force limits apply, and who is watching.
• Keep critical moments, such as a stage reveal, on rehearsed routines with a fallback plan.
• Expect claims of learned behaviour to be backed by a live test at the venue rather than a video.

Frequently asked questions

What is a vision-language-action model in simple terms?

It is an AI model that looks at camera images, reads an instruction in everyday language and outputs the movements a robot should make. It usually starts from a model trained on internet text and images, then learns from recordings of robots performing tasks. The aim is one model that can handle many tasks and objects without an engineer programming each movement separately.

Are rental robots at events controlled by VLA models today?

Generally not, based on what the developers have published. Models such as Helix, π0 and Gemini Robotics are shown mainly in research or company demonstrations, and some are released only to partners. Robots hired for events typically use pre-programmed routines combined with live remote control. Ask the supplier directly how a specific robot is controlled for your booking.

How is a VLA model different from a chatbot on a robot?

A chatbot produces text or speech, so it can let a robot hold a conversation, but it does not move the arms or hands. A VLA model outputs motor commands, so the instruction and the camera view directly shape the physical movement. Some robots combine both: a language model for conversation and separate software or a learned model for motion.

Do VLA models make robots safe around people?

No model alone makes a robot safe. Developers such as Google DeepMind describe combining the model's own reasoning about safe actions with separate low-level safety systems. At events, safety still depends on physical limits, emergency stops, trained operators, barriers where needed and site rules. This is general information, not legal advice; check requirements with your venue and insurer.

How this article was made: written by the humanoid4rent editorial team with AI assistance; every factual statement was checked against the sources listed below on 2026-10-31. Spotted an error? Write to info@humanoid4rent.com.