humanoid4rent
AccessoRichieda preventivi

Chatting robots: how LLM conversation works, and where latency and guardrails bite

A talking robot chains speech recognition, a language model and a synthetic voice. Each step adds delay and risk. Here is how the pipeline works and why scripted and generative modes behave so differently in public.

Di Redazione di humanoid4rent · pubblicato il 12 novembre 2026

Questa pagina non è ancora stata tradotta
Sta leggendo il testo inglese perché non esiste ancora una versione in Italiano. Non è stato omesso nulla: questa è la pagina intera, nella lingua in cui è stata scritta.

When a humanoid robot holds an open conversation, it usually runs a chain of software: speech recognition turns the visitor's words into text, a large language model (LLM) writes a reply, and text-to-speech gives it a voice. Each step adds delay, and the language model can produce confident but wrong answers. That is why scripted and generative modes behave so differently in front of the public.

This explainer walks through the pipeline, the three limits that matter most at events (latency, hallucination and content filtering), and the transparency rule in the EU AI Act that applies to talking machines.

The basic pipeline: ears, brain, voice

Most conversational robots follow the same broad pattern, whatever the body looks like.

  1. Listening. Microphones pick up sound. Software has to decide when a person has started and stopped speaking, a task called voice activity detection.
  2. Speech recognition. The audio is converted to text, either on the robot or by a cloud service.
  3. Language model. The text, plus instructions written by the operator (often called a system prompt), is sent to an LLM, which generates a reply.
  4. Speech synthesis. The reply is turned back into audio and played through the robot's speaker, sometimes with matching gestures.

A documented example is the study by Kim, Lee and Mutlu presented at the 2024 ACM/IEEE Human-Robot Interaction conference. According to the paper, the team ran the SoftBank Pepper robot with Google Cloud speech-to-text, OpenAI's GPT-3.5 model (text-davinci-003) for the replies and Pepper's own animated-speech module for voice and gesture.

Chained versus speech-to-speech

There is now a second architecture. OpenAI's voice agents documentation contrasts a chained pipeline (speech-to-text, then the model, then text-to-speech) with a speech-to-speech model that hears audio and answers in audio within one session. OpenAI describes the single-model route as lower in latency but offering less control, because there is no intermediate text to inspect. It describes the chained route as slower but more transparent, since each stage can be checked, replaced or filtered before anything is spoken.

For a robot in a public space, that trade-off is not academic. The chained design is the one that lets an operator run a policy check on the reply before the robot says it aloud.

Latency: why the robot pauses

Human conversation runs on very short gaps. A comparative study of ten languages by Stivers and colleagues, published in PNAS in 2009, found a general avoidance of overlapping talk and of silence between turns, with differences between the languages' average gaps falling within a range of about 250 milliseconds. People are therefore sensitive to pauses that a machine finds hard to avoid.

In a chained robot, the delay adds up from several sources:

  • End-of-speech detection. The system must wait long enough to be sure the visitor has finished. OpenAI's voice activity detection guide explains that its default mode chunks audio on periods of silence, while a newer semantic mode uses a classifier to judge from the words whether the person has finished. An "eagerness" setting trades faster replies against the risk of cutting people off.
  • Network round trips. Cloud speech recognition and cloud LLMs depend on venue connectivity, which is often congested at busy events.
  • Generation time. Longer answers take longer to produce, and the robot cannot start speaking until at least part of the reply exists.
  • Speech synthesis and gesture planning. These add a further step before sound comes out.

Timing problems also affect how people behave. In the Kim et al. study, participants commented on difficulty knowing when they could speak to the voice-only agent, with one calling it choppy and disconnected. The paper did not report response times as a measured result, so it is not evidence of a specific delay, but it shows that turn-taking is part of how people judge the experience.

Noise makes everything harder. A crowded hall degrades speech recognition, which feeds wrong text to the model, which then answers a question nobody asked. Our guide on speech, languages and noisy venues covers the practical side of microphones and languages.

Hallucination: fluent but wrong

Language models generate plausible text; they do not look facts up unless they are connected to a source that does. A widely cited survey by Ji and colleagues in ACM Computing Surveys describes how neural text generation is prone to producing unintended content, including in dialogue systems, and reviews ways to measure and reduce it. No method in that literature removes the problem entirely.

At an event this shows up in mundane ways: a robot that invents the time of a keynote, names the wrong stand number, gives a product price that does not exist, or makes up details about itself. Because the voice is confident and the body is human-shaped, visitors may trust the answer more than they would a web search.

Common mitigations include:

  • Restricting the model to a short, approved information sheet about the event and instructing it to say it does not know when the answer is not there.
  • Keeping replies short, which reduces both delay and the room for error.
  • Routing high-stakes questions (medical, safety, directions to exits) to staff rather than the robot.

Guardrails and content filtering

Hallucination is about accuracy. Content filtering is about what the robot should never say or agree to do. Research suggests this is a real gap. Hundt, Azeem, Mansouri and Brandão, writing in the International Journal of Social Robotics in 2025, tested several LLMs in human-robot interaction scenarios and reported discriminatory outputs across protected characteristics, as well as responses that accepted dangerous, violent or unlawful instructions when given open-ended prompts. The authors call for thorough risk assessment before such systems are deployed.

For a public robot, practical guardrails usually sit in layers: a system prompt that sets tone and topics, a filter on the transcribed input, a filter on the generated reply before it is spoken, and a human operator who can mute or take over. None of these layers is perfect on its own. Visitors, especially groups, will test a talking robot with provocative questions, and that is worth planning for.

A separate point: in many conversational demonstrations, a person is supplying or steering the speech remotely. Our guide to teleoperated and autonomous robots explains what audiences actually see in each case.

Scripted and generative modes behave differently

Before LLMs, social robots relied on authored dialogue. Aldebaran's QiChat and ALDialog documentation for NAO and Pepper describes topics made of rules, each linking an expected human input to a set robot answer, with concepts that group synonyms and fallback rules for unmatched input. The robot can only say what someone wrote, and it only responds when it recognises a pattern.

That contrast explains most of what organisers notice:

  • Scripted mode is predictable, fast to respond (no generation step) and easy to approve in advance with legal or brand teams. It feels rigid when visitors go off script.
  • Generative mode handles unexpected questions and feels more natural, but replies are slower, vary each time and cannot be fully approved in advance.

User research points to a further effect. Kim et al. found that the LLM-powered robot raised participants' expectations of sophisticated non-verbal behaviour and did well at connection-building and deliberation, but performed worse than the text and voice comparisons on logical communication and could make some participants anxious. A fluent voice in a humanoid body sets expectations that the rest of the robot may not meet.

Many deployments therefore combine the two: scripted lines for welcome messages, product claims and safety information, with a constrained generative mode for small talk. Preparing that content takes time; see our guide on custom speeches and lead times.

Transparency: the AI Act's Article 50

In the EU, Article 50 of the AI Act sets transparency duties. According to the European Commission's AI Act Service Desk, providers must design AI systems intended to interact directly with people so that those people are informed they are interacting with an AI system, unless that is obvious to a reasonably well-informed, observant and circumspect person in the context. A separate paragraph requires providers of systems that generate synthetic audio, image, video or text to mark outputs in a machine-readable way.

Whether a humanoid robot is "obviously" a machine may seem clear, but a voice that sounds like a person, or a robot whose replies are partly human-operated, raises harder questions. Our article on Article 50 and talking robots looks at this in more depth, including timing. This section is general information, not legal advice.

What this means for event organisers

  • Decide the mode first. Ask whether the robot will run scripted dialogue, generative conversation, remote operation or a mix, and which content falls under each.
  • Ask about the pipeline. Where speech recognition and the language model run (on the robot or in the cloud) affects delay, Wi-Fi needs and what happens to visitors' voices.
  • Give the model a fact sheet and limits. Short approved information, short replies and a clear "I don't know, please ask our staff" route reduce invented answers.
  • Check the guardrails and the human. Ask how inputs and outputs are filtered and who can mute or take over the robot during the event.
  • Put it in the brief. Include languages, noise levels, sensitive topics and approval steps when you brief a robot supplier.

Frequently asked questions

Why do talking robots pause before they answer?

A typical robot must detect that you have stopped speaking, convert your speech to text, send it to a language model, wait for the reply and then synthesise speech. Each stage takes time, and cloud services add network delay, which is often worse on busy event Wi-Fi. Human conversation runs on very short gaps between turns, so even a pause of a second or two is noticeable to visitors.

Can a robot running an LLM give wrong information to visitors?

Yes. Language models can generate fluent answers that are not true, a problem researchers call hallucination. At events this can mean wrong times, stand numbers or product details. Restricting the model to an approved fact sheet, keeping replies short and sending important questions such as safety or medical matters to staff reduce the risk, but no current method removes it entirely.

What is the difference between scripted and generative robot conversation?

In scripted mode, someone writes rules that link expected questions to fixed answers, so the robot is predictable and quick but rigid when visitors go off script. In generative mode, a language model writes each reply on the spot, which handles unexpected questions but is slower, varies each time and cannot be fully approved in advance. Many deployments combine the two.

Does the EU AI Act require a robot to say it is an AI?

Article 50 of the AI Act requires providers to design AI systems that interact directly with people so that those people are informed they are dealing with an AI system, unless this is obvious from the context. How that applies to a particular robot, voice or partly human-operated setup depends on the facts. This is general information, not legal advice.

How this article was made: written by the humanoid4rent editorial team with AI assistance; every factual statement was checked against the sources listed below on 2026-11-12. Spotted an error? Write to info@humanoid4rent.com.

Fonti

I fatti riportati in questo articolo sono stati verificati su queste pagine nel giorno in cui è stato scritto.

  1. Understanding Large-Language Model (LLM)-powered Human-Robot Interaction – arXiv (HRI '24)
  2. Voice agents – OpenAI API documentation
  3. Voice activity detection (VAD) – OpenAI API documentation
  4. Universals and cultural variation in turn-taking in conversation (PNAS, 2009) – Europe PMC
  5. Survey of Hallucination in Natural Language Generation – arXiv (ACM Computing Surveys)
  6. LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actions – arXiv (International Journal of Social Robotics)
  7. QiChat and ALDialog – Aldebaran NAOqi documentation
  8. Article 50: Transparency obligations for providers and deployers of certain AI systems – AI Act Service Desk, European Commission

Altri contenuti dal blog

Desiderate un robot al Vostro prossimo evento?

Descrivete una sola volta la data, il luogo e ciò che il robot dovrà fare. I fornitori risponderanno con preventivi che potrete confrontare in modo omogeneo.

Invia una richiesta

Lavorate nella robotica? Scrivete per il nostro blog. →