Agent loops

An agent that decides for itself when the work is done will find ways to make the check pass without doing the task. METR reports this across frontier models rather than at one lab, and most often when the tests are hidden from the agent.

What people are trying

Hiding test cases and scoring code from the agent, running the scorer somewhere the agent cannot read or write, and using a second model to flag runs that scored suspiciously well. Hardening the environment works where telling the agent not to cheat does not.

• Recent Frontier Models Are Reward Hacking · METR · read 09/19/2026

  The most recent frontier models have engaged in increasingly sophisticated reward hacking, attempting (often successfully) to get a higher score by modifying the tests or scoring code, gaining access to an existing implementation or answer that's used to check their work, or exploiting other loopholes in the task environment.

• Frontier Risk Report (February to March 2026) · METR · read 09/19/2026

  attempted to reward hack in ~80% of attempts on tasks in an early version of MirrorCode, when test cases were hidden from the agent.

Where it bites: Coding agents

Ask a person to approve every action and they stop reading what they approve. Decide it automatically and you take an error rate instead. Anthropic published the false negative rate of its own classifier for this and says it has not found a way to close the gap that is worth what it costs.

What people are trying

Scoring each action for risk and for whether the person's own words already cover it, tuned to catch the actions that cannot be undone. The remaining gap has resisted prompt engineering.

• How we built Claude Code auto mode: a safer way to skip permissions · Anthropic · read 09/19/2026

  Over time that leads to approval fatigue, where people stop paying close attention to what they're approving.

• How we built Claude Code auto mode: a safer way to skip permissions · Anthropic · read 09/19/2026

  The 17% false-negative rate on real overeager actions is the honest number.

Where it bites: The agent harness

A skill is instructions and scripts the agent starts following once it decides the skill applies, and there is no automated way to tell a safe one from an unsafe one. The maker's own advice is to audit every file by hand, the way you would before installing software.

What people are trying

Reading the whole bundle for network calls and file access that do not match what the skill claims to do, and treating anything that fetches from an outside URL as the riskiest kind, because a skill that was safe when installed can stop being safe when what it fetches changes. Content scanning exists in places and does not cover every way a skill arrives.

• Agent Skills · Anthropic (Claude Platform Docs) · read 09/19/2026

  Even trustworthy Skills can be compromised if their external dependencies change over time

Where it bites: Skills

A voice agent listening for you to break in has to decide, in real time, whether the sound it just heard was you. It stops mid-sentence for audio that turns out to contain no words, which a caller experiences as the assistant losing its thread for no reason.

What people are trying

Moving from a fixed silence threshold to interruption handling that reads the audio for intent, and building a way back: once the interruption turns out to have produced an empty transcript, resume from where the sentence stopped.

• Turns overview · LiveKit · read 09/19/2026

  In some cases, the framework detects human speech audio and interrupts the agent, but the transcription comes up empty as no actual words are spoken.

Where it bites: Voice agents