Guardrails

Guardrails are checks on what goes into a model and what comes out: an input filter over the user’s own message or retrieved content, an output validator over what the model produced, a separate classifier model trained to say whether something is safe, a schema check that an answer is well-formed, and an allowlist of which actions are even reachable. OWASP’s own mitigation list for prompt injection names one form of this directly: “Apply semantic filters and use string-checking to scan for non-allowed content”[1].

A guardrail is a probabilistic check: it decides that something looks disallowed, usually with a model of its own, and it can be wrong in both directions. It is not the control that holds. The control that holds is a check in code that tests a specific fact and refuses the action when the fact is false, whatever anything upstream concluded: safety’s own example is exactly that check. A guardrail lowers how often that check gets tested; it never stands in for it.

This topic is not a level on the ladder; it applies at every level, the way safety does. It is also a decision made on every request, so its cost and its delay are paid every time.

This page is sourced, not measured: what each kind of guardrail catches comes from primary sources, and none of them has been run against real traffic here and scored.

def _delimited(note: str) -> str:
    """Marks the boundary of untrusted content in the prompt. Delimiting is a hint to the model,
    not a guarantee: the permission check below is what actually stops an unauthorized action,
    the way this page's Use it lane and rag's own "prompt injection through retrieved text"
    failure mode both say."""
    return f'<retrieved-note untrusted="true">\n{note}\n</retrieved-note>'

Failure modes

At each level

• Direct prompting: an input filter on the user’s message and an output check on the answer are the only two points that exist yet. These are the same two halves chat’s single call has.
• Added context: a retrieval rail becomes relevant for the first time, since retrieved content is now something a check can inspect before it reaches the model. That is exactly the failure mode RAG names for prompt injection through retrieved text.
• Tool use: an execution rail (a check on what a tool call is about to do, before it runs) is where the code-side permission check safety’s own example builds actually lives; this is the level where a guardrail stops being advisory and starts standing in front of a real action.
• Agent loops: checks that ran once per exchange at lower levels now need to run on every step of a loop with no fixed length, the same shift a single agent’s own step cap responds to for cost: a guardrail evaluated only at the start of a run cannot catch what the run drifts into by its twentieth step.
• Teams of Agents: one agent’s output becomes another agent’s input, so a check placed only at the system’s outer boundary misses content moving between agents entirely: the reason agent graphs checks a handoff against an allowlist before acting on it.
• Always-on agents: nobody is reading each decision as it happens, so the signal is the refusal rate over time rather than any single refusal: a rate that moves means either the traffic changed or the check did, and both are worth knowing about.

Practices

• Say out loud, for each check, whether it is a guardrail or a control. If the sentence that justifies running an action unattended names a classifier, the system has no control yet.
• Measure both rates. A catch rate on attacks without a refusal rate on legitimate requests is half a number, and it is the half that always looks good.
• Put each check where it can see what it is judging: retrieved text before the model reads it, a tool call before it runs, an answer before it is returned.
• Log the decision and the reason, not just the outcome, so a refusal rate can later be split into correct and incorrect rather than only counted.
• Attack your own checks on a schedule rather than when something goes wrong. Red teaming is how a guardrail’s real precision gets found before a user finds it.

Run it

Try it

1. Use it

   Think of a legitimate request a product has refused you. Write down which of the five points a check could have run at (input, dialog, retrieval, execution, output) the refusal most likely came from, and what the refusal message did and did not tell you about why. A refusal that explains nothing costs the same as a wrong one.

2. Build it

   Open examples/safety/run.py and read _delimited alongside _permitted. Write down, for each, which of the five rail types NeMo Guardrails names (input, dialog, retrieval, execution, output) it corresponds to, and which of the two still holds if the model ignores everything it was told.

3. Either lane

   Pick one guardrail tool named on this page. Read its own documentation for one thing it explicitly says it does NOT do or does not guarantee, and write down what would have to catch that gap instead.