Structured output

Structured output means the reply comes back in a fixed shape (a JSON object with named fields), not a paragraph your code has to parse by guessing. Makers reach it two ways. A JSON-Schema response format constrains which tokens the model may produce next, so OpenAI says a model given one “will always generate responses that adhere to” it, and lists among the benefits “No need to validate or retry incorrectly formatted responses”[1]; Gemini takes the same approach through a schema in response_format[2]. Anthropic supports schema-constrained JSON responses through output_config.format, and separately supports strict tool inputs through strict: true. These can be used independently or together[3]. A direct structured reply without tool execution sits at level 1, one request and one response, in a shape your code chose first.

OpenAI also lists cases where a reply still may not match: a refusal, or a response cut short by the token limit[1]. And no schema check confirms the values are right. Validating your own side and retrying once is still worth doing, the way Pydantic raises “an error with a breakdown of what was wrong”[4].

A model announced in September 2026 takes the idea further: Jev, from TypeSafe AI, generates no text at all, only what TypeSafe calls “typed probabilistic decisions”[5]. It is in early access behind a waitlist, and its published figures are TypeSafe’s own, unmeasured here.

This page provides primary references and illustrative examples. The examples are scripted, not measured model runs; source references do not establish the correctness of every implementation or outcome.

"What is the DW-480's warranty?"

def run(question: str, model: Model, tracer: Tracer, *, corpus_dir=DEFAULT_CORPUS_DIR) -> Answer:
    match = APPLIANCE_RE.search(question)
    appliance = match.group(0) if match else "DW-300"
    sections = load_sections(corpus_dir)
    passage = "\n\n".join(sections[cite].text for cite in WARRANTY_SECTIONS)
    tracer.record(kind="code", decided_by="code", title="Find which appliance the question asks about", detail=appliance)
    messages = [
        Message(role="system", content=SYSTEM_PROMPT),
        Message(role="user", content=f"{passage}\n\nAppliance: {appliance}"),
    ]
    record: object = {}
    for attempt in range(MAX_RETRIES + 1):
        completion = model.complete(messages, schema=SCHEMA, max_tokens=200)
        tracer.record(
            kind="model",
            decided_by="code",
            title="Ask the model for JSON" if attempt == 0 else "Ask again with the validation error",
            detail=completion.text[:200],
            tokens_in=completion.tokens_in,
            tokens_out=completion.tokens_out,
            ms=completion.ms,
        )
        try:
            record, problems = json.loads(completion.text), None
            problems = _validate(record, appliance)
        except json.JSONDecodeError as exc:
            record, problems = {}, [f"invalid JSON: {exc}"]
        tracer.record(kind="code", decided_by="code", title="Validate against the schema", detail="; ".join(problems) or "valid")
        if not problems:
            return Answer(text=json.dumps(record, sort_keys=True), citations=WARRANTY_SECTIONS)
        if attempt < MAX_RETRIES:
            messages.append(Message(role="user", content=f"That did not validate: {'; '.join(problems)}. Reply again with corrected JSON only."))
    return Answer(text=json.dumps({"error": "did not validate after retry", "last": record}), citations=[])

python -m examples.structured_output --model stub:scripted

Failure modes

Cost and latency

Illustrative, not measured: no result file exists for this technique yet, so every number below is a worked example. See docs/EVALS.md for how a real run is recorded and scored.

How to Evaluate It

Structured output adds a check no plain-text reply can be given: whether the reply is valid against its schema at all, tracked separately from whether the values in it are right. A run can score well on validity and badly on the values, or the reverse, and the two numbers together say more than either alone. Count the retries as a third number. A schema that needs a second attempt on a third of its inputs is a schema to rewrite, not a model to replace.

This example extracts a fixed warranty record rather than answering the question it is handed, so scripts/eval_run.py will not score it against the site’s 60-question set; asking prints that reason and stops (docs/EVALS.md). The set that would measure it is a different one: a list of passages, each with the record it should produce. Score valid-JSON rate, accuracy field by field against those records, and how often the retry was needed.

Run it

Try it

1. Use it

   Find a feature that turns text into a form or a calendar event and give it an ambiguous input. Does it ask you to confirm, or commit to a guess?

2. Build it

   Run python -m examples.structured_output --model stub:scripted from the repo root: the first record is rejected for a warranty term written as a word; the retry validates. Now rename one field in REQUIRED_FIELDS in examples/structured_output/run.py: both passes fail the same way, and the run ends with an error saying it did not validate after retry, capped at one.

3. Either lane

   Write the schema you would want for a task in your own life, a recipe or a receipt, before asking a model to fill it. Which fields are truly required, and which would you rather leave blank than guessed?

4. Build it

   Open the accuracy specs from the manual recipe and find the schema for a row of its table. Which fields would a person have to check against the manual before the row could be trusted?