Searching your documents and giving the results to the model.
Measured
How it works · conceptual architecture
Indexing prepares the sources. A query selects evidence for this answer.
Versioned text with access rules
Split, preserve IDs, index the content
What the user needs to know
Lexical, vector, hybrid; optional rerank
Question + selected evidence
Supported claims and valid citations
Apply document permissions. Keep source IDs and revision metadata.
Check whether the selected evidence covers every part of the question.
A real citation ID can still support the wrong claim. Check entailment as well as existence.
Same concept, different task and consequences. Switching starts a fresh walkthrough; prior answers and approvals do not carry over.
Retrieving external information and placing it in the model's context to support an answer.
Follow a question through source retrieval into a grounded answer. Inspect whether the retrieved passages actually support the response, including exceptions and unanswered parts.
Is water damage covered by the DW-480 warranty?
Visible query, retrieved passages, grounded answer, source links, and an abstention when evidence is insufficient.
The setting makes the example concrete. Carry the underlying pattern into your own work; adapt the sources, tools, and level of oversight to your task.
Is water damage covered by the DW-480 warranty?
Authored case. Select any record below; nothing is sent to a model.What changed: Duration and exclusions are distinct pieces of evidence.
The collection may be incomplete or outdated. A citation is useful only when its passage supports the associated claim.
Describe your task to your own model and use Retrieval-augmented generation (RAG) as a reference. Ask whether it fits, which alternatives meet the same automation needs, and how you would implement and check the result.
Retrieval-augmented generation, or RAG, supplies retrieved information to a model when it generates an answer. Retrieval can use keywords, embeddings, or both; the sources may be your documents or another searchable collection. The 2020 RAG paper describes combining retrieval with generation so the model can draw on external information[1].
This page teaches a simple, fixed retrieval pipeline: split documents into passages, embed them, retrieve a few relevant passages for the question, and send those passages to one model call. The instruction is to answer from the evidence, but the model can still make unsupported claims. The fixed number of passages and single generation call are choices in this example, not rules that define all RAG systems. Other implementations rerank, rewrite queries, retrieve repeatedly, or combine evidence across documents.
The simple pipeline sits at level 2 because code determines how context is selected. If a model chooses successive searches, this site calls that agentic RAG. Both approaches augment generation with retrieved information.
This page is measured: the cost and the score under How to Evaluate It come from a recorded run of this example on a real model, and hold for that model’s class. The step-through just below is still a scripted illustration, and source references do not establish the correctness of every implementation or outcome.
This separate, scripted trace illustrates the code example discussed in the implementation details. Use it to inspect individual steps, inputs, and outputs.
Search the documents, give the closest passages to the model, and ask once.
This is a scripted illustration, not a recorded model run. Its timing and token figures are not measured performance.
"How long is the warranty on the DW-480, and what voids it?"
Upload files to a chat app’s project or file feature and ask about them. The product may put the files directly into context, retrieve selected passages, or combine both approaches. Uploading a file alone does not tell you which method it uses; check the product documentation.
Ask questions a handful of passages can answer on their own. “What does the warranty cover” is a focused lookup. “Summarize every change across all our contracts this year” requires much broader coverage: a few highly ranked passages may omit important changes. For that task, check whether the product can systematically cover the collection. Narrower questions and an explicit document checklist make omissions easier to notice.
Anthropic’s own documentation describes this directly: once a project’s uploaded files approach what the context window can hold, Claude switches into what Anthropic calls RAG mode. Anthropic says that expands how much a project can hold by up to ten times while maintaining response quality[2]; that is the maker’s claim, not a number this site has measured. OpenAI documents the same pattern for its own file search feature: it retrieves passages from uploaded files by keyword and meaning together, and returns an answer with citations to the files it used[3].
Check the citations every time the product shows them. Open the source it names and confirm the sentence it cites is actually there. A citation that does not obviously support the sentence beside it, or an answer with none at all, is unconfirmed. It does not by itself tell you whether retrieval failed, the model ignored evidence, or the interface omitted the citation.
RAG can combine facts from multiple documents when the necessary evidence is retrieved and used correctly. A single search can return passages from several files, but questions whose second lookup depends on the first answer may need query decomposition or repeated retrieval. Inspect all required sources and compare a fixed pipeline with an agentic one on the same task.
The minimal version of RAG is four fixed steps: chunk the documents, embed the question and every chunk, keep the top few chunks by similarity, and ask the model once with those chunks as its only sources.
Chunking splits documents into pieces small enough to embed and retrieve individually. The example below chunks by section, since the synthetic document set already has numbered sections; a real document set usually needs its own splitter, tuned so a chunk holds one complete idea rather than cutting a sentence or a table row in half.
Embedding turns text into a vector, a fixed-length list of numbers, using a model trained so
that texts with similar meaning get vectors that point in similar directions; the mechanism, and
the search built on it, is embeddings and
search. The code below is
written against an Embedder interface with two implementations: a deterministic stub for
tests, and a real embedding model behind the same interface, so the retrieval logic never has to
know which one is running.
Retrieval scores every chunk’s embedding against the question’s embedding by cosine similarity
(how closely the two vectors point in the same direction) and keeps the top k, four by
default. _cosine below returns a plain dot product rather than a full cosine, because both
embedders return unit vectors, for which the two are the same number. This is the one place a
real system usually adds more: a second, more expensive reranking pass over a larger first cut
of candidates, scored by a model trained for that job. Cohere and Jina AI both sell one. The
example skips reranking to keep the pipeline to four fixed steps.
Prompt assembly numbers every retrieved chunk, includes its citation (file#section), and the
system prompt instructs the model to answer using only those sources and to name which ones it
used. Citations are then parsed back out of the model’s answer with a regular expression, so
the calling code always knows, in a form it can check automatically, which sources actually
contributed to the answer.
The same four steps work over an engineer’s own documents, not just reference text. Orbeck Power Systems’ SRB-5030 datasheet states one maximum input voltage; a later engineering change notice supersedes it for two of the board’s three revisions, over a capacitor derating rule, and the datasheet is never reissued to say so. One search that retrieves both documents returns an answer with citations a reader can check by hand, in a production test or in a low-volume engineering bring-up alike. The model reports which document says what; it never assembles the number, the margin, or the verdict on which revision is safe.
Here is the whole pipeline, constants first, as the example runs it:
LEVEL = 2
TOP_K = 4
SYSTEM_PROMPT = (
"You answer questions about Halvorsen appliances using only the numbered sources below. "
"If the sources do not contain the answer, say so instead of guessing. End your answer with "
"a line starting 'Sources:' listing the citations, like 'dw300-manual#3', that you used."
)
def _cosine(a: list[float], b: list[float]) -> float:
dot = sum(x * y for x, y in zip(a, b))
return dot # StubEmbedder and OllamaEmbedder both return unit vectors, so dot == cosine
def _retrieve(question: str, sections: dict[str, Section], embedder: Embedder, k: int) -> list[Section]:
ordered = list(sections.values())
vectors = embedder.embed([question] + [f"{s.title}\n{s.text}" for s in ordered])
query_vec, chunk_vecs = vectors[0], vectors[1:]
scored = sorted(zip(ordered, chunk_vecs), key=lambda pair: _cosine(query_vec, pair[1]), reverse=True)
return [section for section, _ in scored[:k]]
def _build_prompt(question: str, sources: list[Section]) -> str:
blocks = "\n\n".join(f"[{s.cite}] {s.title}\n{s.text}" for s in sources)
return f"Sources:\n\n{blocks}\n\nQuestion: {question}"
def run(
question: str,
model: Model,
embedder: Embedder,
tracer: Tracer,
*,
corpus_dir: Path = DEFAULT_CORPUS_DIR,
top_k: int = TOP_K,
) -> Answer:
sections = load_sections(corpus_dir)
tracer.record(kind="code", decided_by="code", title="Chunk corpus", detail=f"{len(sections)} sections")
sources = _retrieve(question, sections, embedder, top_k)
tracer.record(
kind="code",
decided_by="code",
title="Embed and retrieve top-k",
detail=", ".join(s.cite for s in sources),
)
prompt = _build_prompt(question, sources)
messages = [Message(role="system", content=SYSTEM_PROMPT), Message(role="user", content=prompt)]
tracer.record(kind="code", decided_by="code", title="Build prompt with sources", detail=f"{len(sources)} sources")
completion = model.complete(messages, max_tokens=500)
tracer.record(
kind="model",
decided_by="code",
title="Ask the model for a cited answer",
detail=completion.text[:200],
tokens_in=completion.tokens_in,
tokens_out=completion.tokens_out,
ms=completion.ms,
)
citations = cited_sources(completion.text)
tracer.record(kind="code", decided_by="code", title="Parse citations", detail=", ".join(citations) or "none")
return Answer(text=completion.text, citations=citations, retrieved_sources=[s.cite for s in sources])
Every step above is decided by code, not by the model. The one model call answers the question; it does not choose what happens next, because there is nothing left to choose. Run it yourself:
python -m examples.rag --model stub:scriptedTry level 0, no model at all first if the documents are small enough for plain keyword search or a regular expression to answer the question directly, with no model and no embeddings to keep in sync.
Try putting the whole document set straight into the prompt instead of retrieving from it, if it comfortably fits the model’s context window and you are not reusing the same documents across many separate questions. That is context engineering.
Move up to RAG once the documents are too large, too numerous, or reused too often for either of those to still make sense.
Measured: averages over the 60-question run on Muse Glimmer 30B, a model in the Large local (about 30B) class, on one local GPU. Tokens out include the model's hidden reasoning, which it spends before answering. Holds for this model class only.
One model call per question, always: the pipeline’s code makes exactly one. Agentic RAG, the level 5 version, makes several; both were run on the same questions and model, and the line above compares them.
The site scores every technique against the same 60-question synthetic set, 12 questions in
each of five kinds, over the appliance document set in evals/corpus/. RAG is graded the same
way every other level is: exact match or a rubric where exact match doesn’t apply, plus
citation hit rate, the share of questions where every source the grading rule expects was
actually cited in the answer.
Two kinds matter most for RAG specifically. Multi-hop questions need two chunks retrieved and used together, which is exactly what single-pass retrieval struggles with. Conflicting-source questions need the answer to notice two chunks disagree, not just cite whichever one the search ranked first.
Agentic RAG (level 5): 55 of 60 on the same questions, same model.
On the site's 60-question set, run 09/23/2026 with Muse Glimmer 30B by Meta, a model in the Large local (about 30B) class. Open weights at 4-bit (Q4_K_M), run on one local GPU through Ollama. The tag is a local build of muse-glimmer:30b.
| Question kind | This page | Agentic RAG (level 5) | Share |
|---|---|---|---|
| Lookup | 12 of 12 | 12 of 12 | |
| Numeric | 11 of 12 | 12 of 12 | |
| Conflicting sources | 10 of 12 | 11 of 12 | |
| Not in the documents | 11 of 12 | 12 of 12 | |
| Multi-hop | 2 of 12 | 8 of 12 |
of the sections the questions need reached the prompt
of the sections the questions need were cited in the answer
code chose every step; the model only wrote the answer
answers the score could not read
questions where the code's step or token budget stopped the loop
Graded by the same model on 28 rubric questions, the rest by exact match. Checked by a person on 09/23/2026: All 14 answers scored wrong were read, and each is wrong by its rubric or pattern. The closest calls: C06 names all three documents that disagree and quotes the bulletin superseding the others, but never says which figure is correct, which its rubric asks for; U10 refuses correctly but assumes a child lock exists. This holds for the Large local (about 30B) class only. Not yet run: Small local (about 8B); Frontier API.
The misses sit where single-pass retrieval predicts. Most multi-hop answers were honest: they said the documents did not cover part of the question, because the one search never reached the second document it needed. The worst miss is the other kind. Asked whether a drive belt is covered eighteen months after purchase, the model found the two-year warranty, never saw the exclusion for wear parts, and said yes. That is the case the page’s failure modes warn about: a fluent answer built only from what retrieval happened to return.
To run it yourself, python scripts/eval_run.py --example rag --model <spec> --dry projects the
cost first; docs/FIRST-LIVE-RUN.md is the full sequence and the checks to read before the score.
Citation hit rate on a sample of real questions, and how often a question comes back with zero retrieved chunks above a similarity floor. Watch retrieval latency and generation latency separately, since a slow answer can come from either half.
This example makes one generation call per question. Cost depends on input and output tokens, model rates, query embeddings, retrieval infrastructure, and how often documents are indexed again. Document embeddings can be reused across questions; no component always dominates.
A document changes and the index isn't rebuilt, so the model confidently answers from stale text. Or a document containing an injected instruction gets indexed and later retrieved and followed.
The question, the retrieved chunk ids with their similarity scores, the final citations, and the full prompt sent to the model, so a bad answer traces back to a retrieval failure or a generation failure without re-running anything.
Open a chat app's file feature with a document you know well, and ask a question whose answer needs two sections. Does it find both, or one?
Run python -m examples.rag --model stub:scripted from the repo root: one call, a grounded answer, one citation. Now ask something the corpus does not cover: --question "What is the DR-520 vent length?". Neither the answer nor the citation moves: the citations line is parsed out of the reply, never checked against what retrieval returned. The write and check page adds that check.
Change the retriever. _retrieve in examples/rag/run.py is the only function that picks passages: it embeds the question and every section, then keeps the top k by cosine. Replace its body with a count of shared words, keep the signature, and run again. Which questions get better, and which get worse?
Cause one of the failure modes above on purpose, using the documents in evals/corpus/.
Pick two documents of your own that can disagree over time: a manual and a later errata sheet, say. Put both into a search tool that shows citations, and ask the question the older one alone would answer wrong. Does the answer cite both, or only the one that sounds authoritative?
13 current examples · Products, tools, and models that demonstrate this concept. A selection, not a ranking.
Retrieval framework
Maker’s documentation Checked 09/18/2026Application framework
Maker’s documentation Checked 09/18/2026Retrieval framework
Maker’s documentation Checked 09/18/2026Research notebook
Maker’s documentation Checked 09/18/2026Answer engine
Maker’s documentation Checked 09/18/2026Workplace search
Maker’s documentation Checked 09/18/2026Files and instructions in a chat app
Checked 09/18/2026Files and instructions in a chat app
Checked 09/18/2026Visual workflow builder
Checked 09/18/2026Workplace assistant
Checked 09/19/2026Workspace assistant
Checked 09/18/2026Reranker
Checked 09/18/2026Reranker
Checked 09/18/2026Retrieve the warranty passages, put them in the model request, and return an answer with source references.
Names listed 09/19/2026. 223 of 223 registry entries have been checked against the maker's own page; the registry marks the rest as unchecked.
Last reviewed 09/19/2026. Pages unreviewed for 90 days are flagged for another pass. Markdown version of this page