Robot teleoperation explained: VR control, latency and the people behind the robot

Robot teleoperation means a person controls a robot remotely, usually by wearing a VR headset or motion-capture sensors whose movements are translated into the robot's joints, while video from the robot's cameras is streamed back. It is widely used for demonstrations and for collecting training data. Its main limits are network delay, operator fatigue and how honestly the set-up is disclosed to audiences.

What teleoperation actually is

In a teleoperated system, the decisions come from a human and the robot carries them out. The robot still runs its own low-level control, such as keeping its balance and moving its motors, but what it does next is chosen by the operator. This is different from a fully autonomous robot, which perceives its surroundings and chooses its actions with on-board software.

A widely cited survey, Teleoperation of Humanoid Robots: A Survey by Darvish and colleagues (published on arXiv in January 2023), describes a general architecture with three parts. The operator side captures what the human does. A retargeting layer converts human movement into commands the robot's different body can follow. A feedback channel tells the operator what the robot sees and, in more advanced systems, what it feels.

In practice, most robots you meet at events sit somewhere on a spectrum between full remote control and full autonomy. Our guide on what audiences actually see with teleoperated and autonomous robots covers how this plays out on a show floor.

How operators control the robot

VR headsets and hand tracking

The most visible method today is a consumer or enterprise VR headset. Unitree's open-source xr_teleoperate project on GitHub is a useful public example. Unitree lists support for Apple Vision Pro, PICO 4 Ultra Enterprise and Meta Quest 3, and for its G1 humanoid as well as the arms of its H1, H2 and R1 robots.

The principle is straightforward. The headset tracks the operator's hands, or handheld controllers, and the software calculates which joint angles would put the robot's hands in the matching position, a step engineers call inverse kinematics. Video from the robot's head cameras is streamed back into the headset, so the operator sees roughly what the robot sees. Unitree's documentation notes that the system runs over a local network, that the default stream is 30 frames per second, and that latency depends on network conditions.

Motion capture and whole-body control

Arm-and-hand control is the easier part. Walking, turning and keeping balance while an operator moves freely is much harder, because a humanoid can fall over if it simply copies human motion. The Darvish survey describes wearable inertial sensors and optical motion-capture systems as common ways to capture the operator's whole body, and notes that mapping that motion onto a robot with different proportions is a research problem in its own right.

Newer research uses learned controllers to bridge this gap. OmniH2O, a project published in June 2024 by Tairan He, Zhengyi Luo and colleagues, trained a whole-body policy so that a humanoid could be steered through a VR headset, from an ordinary RGB camera, or by spoken instructions passed through a language model. The operator supplies target poses and the learned controller works out how the robot moves its whole body without falling.

Why latency is the central limit

Latency is the delay between an operator moving and the robot responding, plus the delay before the operator sees the result. Every step adds to it: capturing the pose, computing the robot's movement, sending commands, moving the motors, capturing camera images, compressing and streaming them, and displaying them in the headset.

The Darvish survey summarises several reasons why this matters:

• Human perception. It notes that people react to visual input in roughly 250–300 milliseconds and to touch and force much faster, at roughly 100–150 milliseconds, so delays are more noticeable once haptic feedback is involved.
• Control stability. It states that even round-trip delays of around 100 milliseconds, which it describes as typical of the internet, can destabilise teleoperation, either because operators overcorrect for errors they see late or, in systems with force feedback, through the control loop itself.
• Move-and-wait behaviour. It recalls early research showing that when delay is added, operators tend to make a move, stop and wait to see the result, which slows everything down.
• Motion sickness. It points out that VR video from a walking robot is not stabilised the way human vision is, and that added delay can make operators feel worse.

For events, the practical consequence is simple: teleoperation works best when the operator and robot share a reliable, low-delay connection, usually on site. A congested venue Wi-Fi network can turn a fluid performance into a hesitant one. Our venue checklist covering space, power and Wi-Fi lists what to clarify in advance.

Shared autonomy: the middle ground

Pure remote control asks a lot of one person. A humanoid operator may need to manage both hands, the head, foot placement and balance at once. The Darvish survey describes how this can lead to clumsy motions and repeated attempts, and presents shared control as a remedy: the robot handles part of the task itself and adjusts or assists the operator's input.

Common forms of shared autonomy include:

1. Autonomous balance and walking, with the operator choosing direction and speed through a joystick or body movement.
2. Assisted grasping, where the operator points the hand at an object and the robot completes the grip.
3. Safety filters that limit speed, block movements into certain zones or stop the robot when contact is detected.
4. Predictive displays and high-level commands, which the survey notes were used when communication links were poor.

Shared autonomy is also how teleoperation feeds into robot learning. The Mobile ALOHA paper from Stanford researchers, published in January 2024, used a whole-body teleoperation rig to record human demonstrations and then trained the robot by imitation. The authors report that with 50 demonstrations per task, combined with existing data, success rates rose by up to 90%. This is the same logic behind the vision-language-action models explained in our article on how RT-2, π0, Helix and Gemini Robotics control robots: teleoperation produces the examples that autonomy is later learned from.

Why disclosure has become a question

Because a well-operated robot can look autonomous, audiences can easily draw the wrong conclusion. The best-known case followed Tesla's We, Robot event in October 2024. TechCrunch reported that the Optimus robots that walked, served drinks and chatted with guests were controlled remotely by humans, citing reporting by Bloomberg and The Verge, a note from a Morgan Stanley analyst, and a video in which one robot acknowledged being assisted by a person. Tesla had not made this clear on stage.

Teleoperation is now also part of commercial product descriptions. When 1X announced its NEO home robot on 28 October 2025, it described owners booking a "1X Expert" to guide the robot through chores it cannot yet do, which helps it learn. The NEO product page also describes owners piloting the robot remotely through an app and VR device, and lists questions about whether an expert can connect at any time. That shows the company expects buyers to ask about remote access and privacy.

The lesson is not that teleoperation is deceptive. Remote control is a legitimate, often safer way to run a robot near the public. The issue arises when audiences, journalists or investors are left to assume autonomy that is not there.

Teleoperation at events: the practical limits

Beyond latency, three limits matter most for live events:

• Operator workload. Whole-body control in a headset is tiring, and an operator wearing a headset cannot see the crowd directly. Suppliers often pair the operator with a spotter. Our guide on who operates the robot and what staff you need explains typical roles.
• Cameras and privacy. Video streamed from the robot to an operator is still video of your visitors. Where it is recorded or sent off site, data protection rules apply; see cameras, microphones and visitor privacy. This is general information, not legal advice.
• Physical limits stay the same. A human operator does not give the robot more strength, battery life or stability than its hardware allows. Safety stops and exclusion zones remain essential.

What this means for event organisers

• Ask how the robot will be controlled. Fully remote, shared autonomy or autonomous scripts: each affects staffing, rehearsal and what you can promise guests.
• Check the network plan. Confirm whether the operator is on site, what connection is used and what happens if it drops.
• Decide on disclosure in advance. A short line such as "this robot is guided by an operator" protects your brand and avoids the kind of reaction seen after the 2024 Optimus demonstrations.
• Clarify video handling. Ask whether operator video is recorded, where it goes and for how long it is kept.
• Put it in writing. Include control mode, operator count and disclosure wording in your request; our guide on what to put in a request to a robot supplier shows how.

Frequently asked questions

Are the humanoid robots at events usually teleoperated?

Many are, at least in part. Conversation, gestures and walking routes are often guided by an operator, while the robot handles balance and motor control itself. Some performances use pre-programmed routines instead. The mix varies by robot and supplier, so the reliable way to know is to ask the supplier directly how each part of the performance is controlled and whether an operator will be present on the day.

How much delay is acceptable in robot teleoperation?

There is no single threshold. A widely cited survey of humanoid teleoperation notes that round-trip delays of around 100 milliseconds, typical of the internet, can already destabilise control, and that people notice delays more when touch or force feedback is involved. For events, this is why operators usually work on site over a dedicated, stable local network rather than over public internet links.

What is shared autonomy in robotics?

Shared autonomy means the human operator and the robot split the work. The operator gives high-level direction, such as where to walk or which object to pick up, while the robot's software handles balance, grip adjustment or safety limits. It reduces operator workload, makes movement smoother and can compensate for some network delay. Most practical humanoid deployments today use some form of it rather than pure remote control.

Should we tell guests that a robot is remotely controlled?

Telling guests is generally the safer choice for trust and reputation. After Tesla's October 2024 event, reports that the Optimus robots had been human-controlled drew wide criticism because this was not made clear. A short, friendly sign or an introduction from the host is usually enough. Whether any legal disclosure duty applies depends on your country and context, so check with your own advisers.

How this article was made: written by the humanoid4rent editorial team with AI assistance; every factual statement was checked against the sources listed below on 2026-10-31. Spotted an error? Write to info@humanoid4rent.com.