Models that control robots and other machines.
Sourced
Concept at a glance
Ending or continuingStop motion when the safety controller requires it, even if the model wants to continue.
A safety controller bounds physical actions; the next sensor reading checks their effect.
A focused engineering & technical work example. Additional perspectives appear where they provide a useful contrast.
Connecting model decisions to physical sensing and action through robot or machine control systems.
Follow a physical task from observation through a proposed motion and a checked outcome. The example separates uncertain perception, planning, and the system that enforces physical limits.
Sort packages into bins in a simulated work cell.
A 2D simulated scene, proposed action, constrained controller decision, uncertain-object stop, and a sim-to-real limitations note.
The setting makes the example concrete. Carry the underlying pattern into your own work; adapt the sources, tools, and level of oversight to your task.
Sort packages into bins in a simulated work cell.
Authored case. Select any record below; nothing is sent to a model.What changed: Establish the facts supplied for this version of the task.
The walkthrough is a text simulation, not a robot controller. Real systems require environment-specific safety engineering and validated low-level control.
Describe your task to your own model and use Robots and machines as a reference. Ask whether it fits, which alternatives meet the same automation needs, and how you would implement and check the result.
An embodied model controls something that can physically hurt someone or break something, which is what makes level 7 different here than anywhere else on this site: the “always-on” part is optional (plenty of robots only move when asked) but the model deciding what happens next without a person checking every step is not, and a wrong step is not just a wrong sentence.
Google DeepMind calls the mechanism plainly: a model that “converts vision and language input into motor control, enabling a robot to take action”[1]: a vision-language-action model, or VLA. NVIDIA’s GR00T is built the same way: “an open vision-language-action (VLA) model for generalized humanoid robot skills,” taking “multimodal input, including language and images, to perform manipulation tasks”[2]. The loop underneath is perceive (read the scene), plan (the model proposes what to do), act, and act is the one step this page insists cannot be the model’s alone.
This page is sourced, not measured: what these systems do comes from their makers’ own papers and documentation, and nothing here has been run on a robot and scored. It is illustrated.
This separate, scripted trace illustrates the code example discussed in the implementation details. Use it to inspect individual steps, inputs, and outputs.
The model proposes a move; a safety envelope that does not read its reasoning decides what actually moves.
This is a scripted illustration, not a recorded model run. Its timing and token figures are not measured performance.
"A part sits at (200, -200, 100)mm from home, inside the operator station."
Ask a maker to show it doing your specific task, not a generalization demo. Google DeepMind’s current robotics model is Gemini Robotics 2[1]; Physical Intelligence says its π0.7 “can follow new language commands and perform tasks that were never seen in its training data” and “can compose and recombine the skills it learned to solve new tasks”, comparing it only against that maker’s own earlier models, not every competitor[4]. A claim about generalizing is not a demonstration of the one task you need done; ask for the second.
Ask whether it is fast enough for the job, not just accurate at it. Physical Intelligence says “Dexterous robot manipulation requires π0 to output motor commands at a high frequency, up to 50 times per second”[3]. A model too slow for a task is not a worse machine; it cannot do the task at all, however correct each single decision is. If the job needs that kind of speed, ask for the number, not a demo video that hides the pace.
Ask what protects a person or the room if the model errs, and expect the honest answer to be hardware, not the model. Figure AI describes “strategically placed multi-density foam to protect against pinch points” and “safeguards at the Battery Management System (BMS), cell, interconnect, and pack levels” on its Figure 03[5]. 1X says NEO’s “Tendon-driven actuators create safe movements” and that “NEO’s joints are covered from outside access, making the surface entirely pinch proof”[6]. Neither is the model checking itself; each is a physical limit that holds regardless, and each is its maker’s own claim about its own machine.
Ask those three (the task, the speed, the hardware limit) before it comes in the door. One more matters once it has: what it keeps between visits, a learned routine, a map of your rooms, recorded video, and where that is stored. 1X says “For complex tasks NEO doesn’t know, an Expert from 1X can remotely supervise its actions at scheduled times to help it learn new abilities and get the job done”[6]: read that as the remote person finishing the job as well as teaching it, so “the robot did this” is not always what happened.
Find the physical stop switch before the first run: a button within reach that cuts power, because stopping it must not depend on the thing you are stopping.
The example is a text-simulated gripper, not a real robot: perceive is a short scene description,
plan is the model calling move(x, y, z, speed), and act is a safety envelope the model never
sees and cannot call. A real system’s safety layer has to be independent hardware or independent
code for the same reason: a model cannot be the check on its own output. NVIDIA’s README does
not describe a safety layer, but it does show how bounded the perceive-plan half is. It gives
GR00T’s architecture as “a combination of vision-language foundation model and diffusion
transformer head that denoises continuous actions”, and lists inference as needing “1 GPU with
16 GB+ VRAM (e.g., RTX 4090, L40, H100, Jetson AGX Thor/Orin, DGX Spark)”[2]: a fixed
budget this example sidesteps entirely by simulating in text.
def envelope(proposed: Move, actuator_log: list[Move]) -> EnvelopeResult:
"""The safety check, independent of the model. `actuator_log` stands in for a real motor
controller: this is the only function in the module that ever appends to it, and it only
does so for a move that already passed every check. Its last entry is also where the arm is
now, which is what the path check measures from."""
invalid = _valid(proposed)
if invalid:
return EnvelopeResult(outcome="refused", move=proposed, reason=invalid)
clamped = Move(
x=_clamp(proposed.x, *BOUNDS["x"]),
y=_clamp(proposed.y, *BOUNDS["y"]),
z=_clamp(proposed.z, *BOUNDS["z"]),
speed=min(proposed.speed, MAX_SPEED_MM_S),
)
start = actuator_log[-1] if actuator_log else HOME
for zone in FORBIDDEN_ZONES:
if _path_enters(start, clamped, zone):
return EnvelopeResult(outcome="refused", move=clamped, reason=f"path to the target crosses forbidden zone: {zone['name']}")
actuator_log.append(clamped)
if clamped == proposed:
return EnvelopeResult(outcome="actuated", move=clamped)
return EnvelopeResult(outcome="clamped", move=clamped, reason="target or speed was outside the workspace envelope")Four things about envelope matter more than the specific numbers, and three of them are
mistakes that are easy to make and hard to see.
It refuses a nonsense number instead of clamping it. _valid runs first, because Python’s min
and max pass a NaN through rather than rejecting it: min(speed, 250.0) returns NaN when
speed is NaN, and nothing in a one-sided speed cap stops a negative speed at all. Both would
otherwise be actuated. A number that means nothing cannot be clamped into a number that means
something, so it is refused.
It clamps bounds and speed but refuses a forbidden zone: clamping a target inside a zone back to the zone’s edge would still be a target inside the zone, so a zone violation is never something a clamp can fix.
The order matters. The function clamps to the workspace bounds before checking zones, so a proposal that is out of bounds on one axis and would clamp into a forbidden zone is still caught. Checking the raw proposal first would have missed exactly that case.
And the zone check runs on the path, not the target. Two targets can each sit outside every zone
while the straight line between them cuts through one, so an endpoint-only check would let a
sequence of individually legal moves sweep the arm through the operator station.
_path_enters clips the segment from the arm’s current position against each axis’ pair of
planes and asks whether anything is left: exact for a box, rather than sampling points along
the line and hoping none of a thin crossing falls between two samples. The current position is
the last move that actually reached the actuator, which is why the same target is allowed from
one place and refused from another. tests/test_example_embodied.py has a test for each of
these four, including the two-legal-endpoints attack.
def run_step(model: Model, tracer: Tracer, *, scene: str, actuator_log: list[Move]) -> EnvelopeResult:
"""One perceive-plan-act step: the model sees a short text description of the scene and
proposes the next move; the envelope decides what, if anything, actually reaches the
actuator log."""
messages = [Message(role="system", content=SYSTEM), Message(role="user", content=scene)]
completion = model.complete(messages, tools=[MOVE_TOOL], max_tokens=100)
call = completion.tool_calls[0] if completion.tool_calls else None
if call is None:
tracer.record(
kind="model", decided_by="model", title="Model proposes no move", detail="(no tool call)",
tokens_in=completion.tokens_in, tokens_out=completion.tokens_out, ms=completion.ms,
)
return EnvelopeResult(outcome="refused", move=Move(0.0, 0.0, 0.0, 0.0), reason="no move proposed")
# The model's decision is recorded before anything is made of it. A proposal the envelope
# cannot even read is still a decision the model made, and a trace that skipped it would
# undercount exactly the steps this site charts.
tracer.record(
kind="model", decided_by="model", title="Model proposes the next move",
detail=", ".join(f"{axis}={call.arguments.get(axis)!r}" for axis in ("x", "y", "z", "speed")),
tokens_in=completion.tokens_in, tokens_out=completion.tokens_out, ms=completion.ms,
)
try:
proposed = Move(
x=float(call.arguments.get("x", 0.0)),
y=float(call.arguments.get("y", 0.0)),
z=float(call.arguments.get("z", 0.0)),
speed=float(call.arguments.get("speed", 0.0)),
)
except (TypeError, ValueError) as exc:
# A tool argument is whatever the model wrote. "far left" is not a number, and letting
# float() raise here would take the controller down instead of refusing one bad move.
# Refusing is the same outcome the envelope reaches for a number it cannot use, and it
# is reached the same way: without actuating anything.
reason = f"move arguments are not numbers: {exc}"
tracer.record(kind="code", decided_by="code", title="Safety envelope: refused", detail=reason)
return EnvelopeResult(outcome="refused", move=HOME, reason=reason)
result = envelope(proposed, actuator_log)
tracer.record(
kind="code", decided_by="code", title=f"Safety envelope: {result.outcome}",
detail=result.reason or f"actuated as proposed: {result.move}",
)
return resultEvery proposed move is decided_by: "model"; every clamp and every refusal is decided_by: "code", and the envelope’s decision never reads anything about why the model proposed a move
— only the numbers. Run it yourself:
python -m examples.embodied --model stub:scriptedTry computer use or a single agent first if the model’s actions can only touch software: a browser, a file, an API. Nothing about controlling a screen needs a safety envelope with physical units in it.
Skip a learned policy entirely, embodied or not, for a machine whose every motion can be written down in advance: a fixed pick-and-place cycle on a factory line does not need a model deciding what to do next, only a program executing a known sequence, which is level 0, no model at all wearing a robot arm.
Move up to an embodied model once the task genuinely varies (the part is never quite in the same place twice, the packaging changes, the room is not laid out the same way from one run to the next) enough that no fixed program covers it, but the safety envelope around it still has to be written and tested like any other piece of code, before the first real move, not after.
Illustrative, not measured: no result file exists for this technique yet, so every number below is a worked example. See docs/EVALS.md for how a real run is recorded and scored.
This example proposes and clamps motor commands in a simulated workspace; it does not answer
questions about the document corpus, so the site’s shared 60-question set does not apply, the
same reason it does not apply to computer use. What
would be measured here: whether any proposed move that should have been refused or clamped ever
reaches the actuator log under adversarial scripting (this should be exactly zero —
tests/test_example_embodied.py checks it on the stub every run), how often a legitimate,
in-bounds move gets refused or clamped by mistake (a false positive costs real task completions,
not just safety), and the wall-clock time from scene to actuated move under load.
The rate of refused and clamped moves against total proposals: a rising rate is worth reading as a sign the model is drifting toward the envelope's edges, not just a number to alert on, and the wall-clock time of the perceive-plan step against the control frequency the task actually needs.
Cost tracks calls per second, which for a real control loop is fixed by the task (up to 50 a second for dexterous manipulation, per Physical Intelligence's own figure for π0), not by how many decisions turn out to matter: most steps are ordinary and cost the same as the ones that trip the envelope.
The envelope and a hardware limit disagree about what is safe, or a policy that only ever saw simulation makes a confident, wrong move against real friction or real sensor noise the simulation never modeled.
Every proposed move, the envelope's decision and why, the move actually sent to the actuator when one was, and the wall-clock time for the whole perceive-plan-act step, so a near-miss can be reconstructed from the log without needing the robot to reproduce it.
Read one robotics maker's safety page, Google DeepMind's for Gemini Robotics or 1X's for NEO, and sort its claims into those about the model and those about the machine. Which would you trust?
Run python -m examples.embodied --model stub:scripted from the repo root: the model asks for 320 mm at 400 mm/s; the envelope clamps it to 300 mm at 250 mm/s. Now change that move in SCRIPTED (examples/embodied/__main__.py) to x=200, y=-200. The target is inside the workspace, and it is refused anyway: the path crosses the operator station. The actuator log stays empty.
Cause a failure mode above on purpose: change FORBIDDEN_ZONES in examples/embodied/run.py to overlap the whole workspace, and rerun the tests to see which start failing.
7 current examples · Products, tools, and models that demonstrate this concept. A selection, not a ranking.
Humanoid robot
Maker’s documentation Checked 09/18/2026Home robot
Maker’s documentation Checked 09/18/2026Humanoid robot
Maker’s documentation Checked 09/19/2026Robotics model
Maker’s documentation Checked 09/18/2026Robotics model
Maker’s documentation Checked 09/18/2026Robotics model
Maker’s documentation Checked 09/18/2026Robotics model
Checked 09/18/2026A model proposes a movement, a controller enforces safe bounds, and sensors report the actual new position.
Names listed 09/19/2026. 223 of 223 registry entries have been checked against the maker's own page; the registry marks the rest as unchecked.
Last reviewed 09/19/2026. Pages unreviewed for 90 days are flagged for another pass. Markdown version of this page