Direct prompting

One well-posed question to a strong model can still come back fluent, confident and wrong, and it is not clear this is a defect that more training quietly removes. One argument from inside OpenAI is that the way models are trained and scored pushes them toward a guess rather than toward saying they do not know.

What people are trying

Changing what benchmarks reward, so that an admission of uncertainty scores better than a confident wrong answer, rather than only adding more hallucination tests on top of scoring that still rewards guessing.

• Why Language Models Hallucinate · arXiv · read 09/19/2026

  the training and evaluation procedures reward guessing over acknowledging uncertainty

Where it bites: Chat

Forcing a schema during decoding makes the shape right every time, and can make the content wrong more often. On small on-device models one measurement found validity rising to every call while answer accuracy fell, which is the opposite of what a constraint is usually assumed to buy.

What people are trying

Letting the model reason without the constraint and applying the schema only at the end, and reporting schema validity and answer accuracy as two numbers rather than one pass rate. Whether the same tradeoff holds on large models is not established.

• The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models · arXiv · read 09/19/2026

  The usual engineering assumption is that hard output constraints improve reliability without changing the underlying answer. We show that this assumption is unsafe for small models.

Where it bites: Structured output

A reasoning model's visible thinking is not a guaranteed account of why it answered as it did. Anthropic slipped hints into questions and found models used them without saying so more often than not, which means the trace you read is evidence about the answer and not proof of it.

What people are trying

Training models to lean harder on their own reasoning, to see whether faithfulness follows. Anthropic's own attempt raised it and then leveled off well short of reliable.

• Reasoning models don't always say what they think · Anthropic · read 09/19/2026

  There’s no specific reason why the reported Chain-of-Thought must accurately reflect the true reasoning process; there might even be circumstances where a model actively hides aspects of its thought process from the user.

Where it bites: Reasoning at answer time

A single image-in call still gets position and counts approximately right, and the makers say so in their own documentation rather than treating it as a rare miss. Anything that needs an exact coordinate or an exact count cannot rest on one call.

What people are trying

Makers document the limit and tell developers to verify a coordinate before acting on it and to design around approximate counts. Where an exact number matters, the answer today is to measure it in code from the image rather than to ask for it.

• Vision · Anthropic (Claude Platform Docs) · read 09/19/2026

  Claude can give approximate counts of objects in an image but might not always be precisely accurate, especially with large numbers of small objects.

Where it bites: Images, audio and video