Letting the model think for longer before it answers.
Sourced
Concept at a glance
Extra reasoning happens while answering; it does not train a new model.
A focused everyday life example. Additional perspectives appear where they provide a useful contrast.
Allocating additional inference computation to work through a problem before returning an answer.
Work through a problem with interacting constraints and inspect the proposed answer against them. The goal is a checkable solution, not a persuasive explanation of how hard the model worked.
Schedule two 45-minute sessions in one room without overlap.
An observable candidate schedule, constraint checker, counterexample, and labeled illustrative cost/quality comparison.
The setting makes the example concrete. Carry the underlying pattern into your own work; adapt the sources, tools, and level of oversight to your task.
Schedule two 45-minute sessions in one room without overlap.
Authored case. Select any record below; nothing is sent to a model.What changed: Establish the facts supplied for this version of the task.
The constraints must be explicit enough to test. More computation does not establish that the model understood an omitted requirement.
Describe your task to your own model and use Reasoning at answer time as a reference. Ask whether it fits, which alternatives meet the same automation needs, and how you would implement and check the result.
Inference-time reasoning is spending more computation after training to get a better answer, without changing the model itself. It comes in two shapes. The first scales one call: extended thinking or a reasoning-effort setting lets the model work through a problem before answering, and Anthropic says that reasoning is billed as output tokens even when the thinking text is not returned to you[1]. The second scales the number of calls instead: ask the same question several times and combine the answers, the way self-consistency samples several reasoning paths and keeps the answer most of them agree on[4].
Anthropic, Google and OpenAI each say close to the same thing about the first kind: more thinking
helps on problems with real intermediate steps and mostly wastes tokens on ones that do not, like
a lookup or a classification[1][2][3]. This page’s example uses the
second kind, since it is the one a fixed StubModel can demonstrate honestly. Either shape is
still level 1: the model deciding what to think about, or which of several samples to trust, has
not changed who decides what happens next.
This page is sourced, not measured: what thinking longer buys comes from the makers’ and researchers’ own papers, and no sampling run here has been scored. It is illustrated.
This separate, scripted trace illustrates the code example discussed in the implementation details. Use it to inspect individual steps, inputs, and outputs.
Ask the same numeric question five independent times and keep the answer most samples agree on.
This is a scripted illustration, not a recorded model run. Its timing and token figures are not measured performance.
"What is the total price to replace the heating elements on both a DW-300 and a DW-480?"
Look for a toggle, a slider, or a menu item labeled “thinking,” “extended reasoning,” or an effort level from low to high, usually near where you type or in settings. Turn it on, or push it higher, only for a question with real multiple steps: a word problem with several dependent parts, a plan that has to account for constraints, a bug you can’t spot at a glance. Anthropic’s own guidance describes what that setting buys: a model that visibly works through a problem, restating what’s being asked, trying an approach, checking it, backtracking if it doesn’t hold up, before giving a final answer[1].
For anything else, leave it off or set it low. Ask a plain factual question, such as what year a law was passed, with the setting on, then again with it off, and time both. If the answer, not just the wait, comes back identical either way, you’ve found a question this setting was never going to help with: the makers’ own guidance says to use minimal effort for fact retrieval and classification, and save the higher settings for coding, math and multi-step planning[2][3].
Some products never show a toggle at all and instead run several attempts behind the scenes, showing you only the one they kept. You can’t switch that off, but you can still check it: ask the same real question again in a brand new conversation, worded slightly differently, and see whether the two answers actually agree. Two confident, different answers to the same question is a sign to verify the fact independently, not to trust whichever one you saw first.
You’ll know the setting earned its cost when turning it on changes the answer on a question you already know the right answer to, not just when it makes the reply read more thorough. A longer, more confident-sounding wrong answer is not a win; check it against something you can verify before trusting the extra length it took to get there.
If an answer is wrong because the model never had a fact it needed, more thinking time will not fix that, no matter how high the setting goes. That’s a missing-information problem, not a reasoning one, and the fix is giving it the fact directly, not asking it to think harder about the same gap.
The example runs self-consistency literally: the same numeric question goes to the model five
times as five independent calls, each reply is asked to end with a line the code can parse
(Answer: <number>), and the code returns whichever number the largest share of the five samples
agree on.
def run(question: str, model: Model, tracer: Tracer, *, n: int = N_SAMPLES) -> Answer:
messages = [Message(role="system", content=SYSTEM_PROMPT), Message(role="user", content=question)]
tracer.record(kind="code", decided_by="code", title="Build one fixed prompt", detail=question)
votes: Counter[str] = Counter()
for i in range(n):
completion = model.complete(messages, max_tokens=200)
answer = _extract(completion.text) or "no answer"
votes[answer] += 1
tracer.record(
kind="model",
decided_by="code",
title=f"Sample {i + 1} of {n}",
detail=completion.text[:200],
tokens_in=completion.tokens_in,
tokens_out=completion.tokens_out,
ms=completion.ms,
)
winner, count = votes.most_common(1)[0]
tracer.record(kind="code", decided_by="code", title="Tally the votes", detail=f"{dict(votes)}")
tracer.record(
kind="code",
decided_by="code",
title="Return the majority answer",
detail=f"{winner} ({count}/{n} samples agreed)",
)
return Answer(text=f"{winner} ({count}/{n} samples agreed)", citations=[])Extracting the final number from free-form reasoning text is its own small, fixed piece of code
(_extract): read from the bottom for a line starting Answer: and pull the number out of it, so
a sample that reasons at length still ends in something machine-checkable. Five samples from a
scripted stub exercise the vote itself, not a claim about how often real models agree with
themselves on a hard question: that claim (whether five samples on a real model land on the
right number more often than one sample does) is exactly what this site’s eval set is built to
measure, once a real run exists (see docs/EVALS.md).
Run it yourself:
python -m examples.inference_time_reasoning --model stub:scriptedEvery step is decided_by: "code": the sample count is fixed, and the code always takes the
majority regardless of what any individual sample said. A model choosing to think longer inside
one call (the other shape of this technique) would still be decided_by: "code" by this site’s
definition too: the code decided to turn thinking on or set an effort level, and the model
deciding what to think about is not the same as the model deciding what the control flow does
next (see docs/EVALS.md).
Skip the extra tokens if a single plain call already gets the answer right on repeat tries: test that before assuming more thinking or more samples will help. And if the model is wrong because it was never given a fact it needed, not because it reasoned badly, more reasoning effort does not fix that; RAG or a better prompt fixes a missing-information problem, not a reasoning one.
Illustrative, not measured: no result file exists for this technique yet, so every number below is a worked example. See docs/EVALS.md for how a real run is recorded and scored.
Self-consistency and extended thinking are graded like every technique here: scored against the
same 60-question set (docs/EVALS.md), with the sample count or the effort level recorded as a
setting on the run rather than a fixed part of the technique. The comparison that actually matters
is one sample against five, or low effort against high, on the exact same questions, since the
whole claim is “more inference-time computation raises the score for some class of model, on some
kinds of question”, and the site’s claim rule requires naming which model class and which kinds
that holds for once a result file exists.
This example samples one arithmetic question five times and returns the majority answer. It
reads no documents and cites nothing, so scripts/eval_run.py will not score it against the
60-question set and says so instead of returning a number measured on the wrong task
(docs/EVALS.md). Measuring it needs questions with a checkable numeric answer, run at one
sample, three and five: the score at each setting, the share of runs where the samples agreed,
and the token cost of each, since five samples cost about five times one.
Agreement rate across samples (unanimous versus split), tracked separately from raw accuracy. A model that agrees with itself confidently and is wrong needs a different fix than one that disagrees with itself but is right on the majority side.
Cost multiplies by the sample count, or by the extra reasoning tokens for a single deeper call, on every question whether or not that question needed it. This is the one technique on the site where the multiplier is a number you choose directly.
A question type that used to have one dominant right answer starts splitting votes evenly after a data or prompt change, and the majority pick becomes close to a coin flip without the interface showing any less confidence than before.
Every sample's raw text and extracted answer, the full vote tally, and which one won, so a bad final answer can be told apart from a bad extraction of an otherwise fine sample.
Find a "thinking" or "reasoning effort" toggle in a chat app. Ask a simple factual question with it on and off, compare the wait and the answer, then a multi-step problem.
Run python -m examples.inference_time_reasoning --model stub:scripted from the repo root: five samples, three agreeing on 79.50, two wrong, and the vote picking the right one. Now edit SCRIPTED (examples/inference_time_reasoning/__main__.py) so all five answers differ: the vote still returns one, reported as 1/5 agreed. The tally, not the answer, says how far to trust it.
Pick a question a model got wrong. Would thinking longer have fixed it, or did it lack the information?
5 current examples · Products, tools, and models that demonstrate this concept. A selection, not a ranking.
Frontier model
Maker’s documentation Checked 09/18/2026Open-weight model
Maker’s documentation Checked 09/18/2026Frontier model
Maker’s documentation Checked 09/18/2026Frontier model
Maker’s documentation Checked 09/18/2026Frontier model
Maker’s documentation Checked 09/18/2026Give the model more reasoning effort or compare several candidate solutions before accepting an answer.
Names listed 09/19/2026. 223 of 223 registry entries have been checked against the maker's own page; the registry marks the rest as unchecked.
Last reviewed 09/19/2026. Pages unreviewed for 90 days are flagged for another pass. Markdown version of this page