Knowledge graphs and GraphRAG

Knowledge graphs connect information. A graph holds facts as entities and the relationships between them: nodes, relationships and properties, in Neo4j’s description of the pattern it sells[2]. “Halvorsen makes the DR-520” becomes two entities joined by a “makes” relationship. Enough facts like that and you can follow a chain of them, hop by hop, from one document to another, instead of needing one passage to state the whole answer. Each hop is a separate, checkable edge: a report quoted on Neo4j’s own page credits that structure with “capturing evidence provenance”[2].

GraphRAG, as Microsoft documents it, builds one automatically: slice the corpus, have a model extract the entities, relationships and claims, cluster the result, summarize each cluster[1]. Extraction is the cost. LazyGraphRAG is Microsoft Research’s own lighter variant, which leaves that model work until a question is actually asked; Microsoft Research states that its data indexing costs are identical to vector RAG and 0.1% of the costs of full GraphRAG[3]. Those are Microsoft’s numbers; the arithmetic is this site’s, and it puts full GraphRAG’s indexing at roughly a thousand times plain retrieval’s.

This is one of three pages in the site’s graph engineering thread: knowledge graphs connect information, while workflow graphs and agent graphs connect work. They sit at level 2: your code decides to extract, and how to walk what comes back. Sourced, not measured: the claims here are checked against primary sources, and no extraction has been recorded and scored.

"What warranty class covers the model that
HLV-5520 fits?"

def run(
    question: str,
    model: Model,
    embedder: Embedder | None,
    tracer: Tracer,
    *,
    corpus_dir: Path = DEFAULT_CORPUS_DIR,
) -> Answer:
    del embedder  # nothing is embedded here; the graph is walked by exact key, not by similarity
    part_number = next(iter(PART_RE.findall(question)), DEFAULT_PART)
    sections = load_sections(corpus_dir)
    graph: dict[str, list[tuple[str, str, str]]] = {}
    for cite in SOURCE_CITES:
        for (subject, relation, obj), source in _extract_triples(cite, sections[cite].text, model, tracer):
            graph.setdefault(subject, []).append((relation, obj, source))
    tracer.record(
        kind="code",
        decided_by="code",
        title="Build the graph from the extracted triples",
        detail=f"{sum(len(edges) for edges in graph.values())} edges over {len(graph)} subjects",
    )
    hop = _two_hop(graph, part_number, "fits", "warranty_class")
    if hop is None:
        tracer.record(kind="code", decided_by="code", title="No two-hop path found", detail=part_number)
        return Answer(text=f"No warranty class found for {part_number} in the graph.", citations=[])
    model_name, warranty_class, cite1, cite2 = hop
    tracer.record(
        kind="code",
        decided_by="code",
        title="Walk the two-hop path",
        detail=f"{part_number} --fits--> {model_name} --warranty_class--> {warranty_class}",
    )
    text = f"{part_number} fits {model_name} [{cite1}], which carries warranty class: {warranty_class} [{cite2}]."
    return Answer(text=text, citations=[cite1, cite2], retrieved_sources=[cite1, cite2])

python -m examples.knowledge_graphs --model stub --question "What warranty class covers the model HLV-5520 fits?"

Failure modes

Cost and latency

Illustrative, not measured: no result file exists for this technique yet, so every number below is a worked example. See docs/EVALS.md for how a real run is recorded and scored.

How to Evaluate It

Multi-hop questions are where a graph is supposed to earn its cost: the site’s shared 60-question set includes twelve of them, each needing two sources joined together, which is exactly what a graph walk does directly instead of hoping a single retrieval happens to surface both passages at once. Citation hit rate on multi-hop questions specifically, compared against RAG’s citation hit rate on the same twelve questions, is the number that would show whether the graph’s cost bought anything here.

No result file exists for this technique yet (see docs/EVALS.md), so this page cannot say a number for any of it. Run python scripts/eval_run.py --example knowledge_graphs --model <spec> --dry to project the cost of a real run before spending anything on one.

Run it

Try it

1. Use it

   Ask a deep-research tool a question that needs two different topics joined together, and look at how it shows its sources. Does it show a connected chain of specific facts, or one flat list of links at the end?

2. Build it

   Run python -m examples.knowledge_graphs --model stub --question "What warranty class covers the model that HLV-5520 fits?" and read the path it prints. Then ask the same question about HLV-7734, which the parts list itself gives no fitment for, and check that the answer reports no path rather than guessing one. Point --model at a real backend to see what changes when a model, not a transcription, does the extracting.

3. Either lane

   Pick one of the failure modes above and try to cause it on purpose: edit one of the two source sections in a copy of evals/corpus/ and see whether the graph the example builds still points at the changed fact or the old one.

4. Either lane

   Sketch the two-hop graph for the SRB-5030 story above: a board revision to the document that governs its input-voltage limit. Then find a document pair you actually work with, an old manual and the change notice that supersedes part of it, and sketch the same shape for it.