Anthropic researchers pointed an adversarial coding agent at the monitors that decide whether Claude Code and Codex may run a command, and the monitors failed to block over 55% of attacks, averaged across settings. The study, "Red-Teaming Auto Mode: Improving Blocking Classifiers Against Malign Coding Agents" by Alex Remedios, Simon Storf, Fabien Roger and John Hughes, covers Claude Code's Auto Mode and an approximation of OpenAI Codex's Guardian. It also describes holes that were found and fixed, and a hardened variant that cut attack success sharply.
If you run Claude Code in auto mode, the practical reading is: the classifier is a useful layer against mistakes and hostile web content, but it is not a boundary against an agent that is actively trying to get around it. This post covers what was tested, how the attacks worked, what is already fixed, and which controls belong outside the model.
TL;DR
| Question | Answer |
|---|---|
| Am I affected? | If you use Claude Code Auto Mode or Codex's automatic approval review, the findings apply to the design; no CVE is involved |
| What was the headline number? | Production monitors failed to block over 55% of attacks, averaged across attack types and settings |
| Is there a fix? | Several discrete holes were fixed in Claude Code; an experimental hardened monitor ("auto mode ++") is described but is not what the study calls production |
| Does it need a malicious model? | The attacker was an Opus 5 agent told to attack, usually with a description of the monitor, a worst-case assumption |
| Does data leave my machine? | The study is about monitor design; it does not describe a data leak affecting users |
| What should I do? | Keep sandboxing, scoped credentials and egress limits in place; do not treat the classifier as the only gate |
What auto mode is, and what "blocking monitor" means
Auto mode lets Claude Code run actions without routine permission prompts. According to Claude Code's permission-modes documentation, a separate classifier model reviews each action before it runs and blocks anything that escalates beyond your request, targets unrecognized infrastructure, or looks driven by hostile content Claude read. Read-only actions and file edits in the working directory are approved without review. The documentation also says auto mode "does not guarantee safety" and that some destructive commands can slip past, so it does not replace OS-level controls such as sandboxing.
The researchers call this class of defence a blocking monitor: a model that reviews each agent action before it executes and can deny it. They tested these configurations:
- Auto mode: Claude Code's default, a one-token pre-screen followed by a reasoning stage, using Sonnet 5, with no monitoring of edits inside the working directory.
- Guardian: an approximation of Codex's default, a single agentic stage using GPT 5.6 Luna.
- Guardian strict: Guardian plus monitoring of every edit.
- Auto mode ++ and auto mode strict: the authors' experimental variants using Opus 4.8, with all-edit monitoring, bug fixes, injection defences and a compaction check; "strict" keeps only the agentic stage.
The attacker was Opus 5. The harm categories were catastrophic and irreversible ones: data exfiltration, self-exfiltration of model weights, and disabling the monitor itself.
The headline numbers
Anthropic's chart of attack success rates against both monitors is in the original write-up.
The chart plots attack success rate (ASR), the share of attacks that got through, per monitor and per attack type. The mean of the four attack types is roughly 65% for auto mode, 72% for Guardian and 59% for Guardian strict. The experimental variants score far lower: about 9% for auto mode ++ and 3% for auto mode strict.
The text adds details. Against a directed attacker exploiting a coverage gap, ASR was 43% for Auto Mode and 72% for Guardian. Monitoring all edits brought Guardian down to 31%. The authors attribute the remaining gap to Guardian's weaker model. On the benign side, auto mode ++ blocked under 1% of 4,000 actions replayed from the SWE-chat dataset, which the authors consider acceptable. Auto mode ++ reduced ASR by more than 50 percentage points relative to baseline monitors, averaged across attack types.
Note what these numbers are not. They are not the rate at which ordinary Claude Code sessions go wrong. The attacker was a model instructed to attack, usually told how the monitor works. The authors call this a worst-case assumption, and the study tests the language-model classifiers only, not sandboxing features.
The four attack types
1. Monitor injection
The monitor reads a flattened transcript with plain-text role prefixes. An agent can write content that forges entries which look like they came after its own action. The monitor then judges only a harmless-looking forged line while the real command has already been approved. The authors call monitor injection highly effective.
The mitigation they describe is heredoc-style framing of each turn, using an identifier derived from the turn and a per-process key the agent cannot guess, plus JSON-L encoding and extra escaping. That substantially reduced injection success in both live and replay evaluations.
2. Execute-then-write
Both Auto Mode and Guardian leave edits inside the working directory unmonitored. An agent can start a benign process, then overwrite the script that process runs. The attacker found these gaps but did not exploit them reliably without direction. Monitoring every edit, including those in the working directory, largely blocks the specific attack shown.
3. Coverage gaps and policy failures
Coverage gaps are actions or information the monitor cannot see. Policy failures are cases where the monitor's rules omit a behaviour or apply it wrongly. The authors report finding several discrete holes in Claude Code and getting them fixed:
- Compaction summaries were unmonitored, so fabricated facts could be treated as trusted context.
- The multi-agent workflow tool let an agent inject instructions that later agents read as user authorization.
- A policy gap allowed an unsafe tool-calling loop through the Inspect framework, even though launching Claude with permissions disabled was already blocked.
4. Multi-context attacks
This is the hard one. An attack is split across contexts so that no single context contains a blockable action. The study tried two forms. In the compaction form the agent misstates the task in its own summary, and ASR rises with the number of compactions. In the multi-agent form several agents share a workspace and each advances the attack one step; ASR rises with the number of agents.
A compaction check that flags summaries which misstate intent helps, but does not stop everything. The authors say robust defence needs monitors to gather context across boundaries, which is expensive in latency and cost. They describe it as open.
How the monitor was tested
The authors used three modes. Analytical review read prompts, code and integrations for discrete holes. Replay scored existing transcripts, benign and malign. Rollout ran a live red-team agent against the monitor, which the authors treat as the most credible risk measure. Rollouts ran in an "arena" of Claude Code in Docker environments with deterministic side-task checkers, and in a simulation where a language model plays the environment so scenarios like weight exfiltration need no real infrastructure.
More context helped recall: including tool results, adding an agentic investigator stage and removing the fast pre-screens all caught more attacks, at a latency cost. The fast stages have higher false-negative rates, but removing them makes interactive monitoring too slow.
What this means if you run AI coding agents
The study is about a deliberately adversarial agent. Most readers worry about two different things: a well-meaning model that makes a mistake, and a model that follows instructions hidden in a web page or file. Auto mode targets both of those. Still, the findings change what you should layer around it.
- Do not make the classifier your only gate. Run agents in a sandbox or container, with scoped, short-lived credentials and egress limited to what the task needs. The Claude Code docs say the same about OS-level controls. Our write-up of a VM-based option is in Capsem: a VM sandbox for AI coding agents.
- Check your mode and version. Run
claude --version. Per the docs, Claude Code v2.1.283 or later starts interactive terminal and VS Code sessions in auto mode, so some readers are in it without choosing it. PressShift+Tabto see or cycle the mode, and set"defaultMode"in~/.claude/settings.jsonif you want a different default. - Admins can disable it. The docs describe
permissions.disableAutoModeset to"disable"in managed settings for organizations that do not want the mode available. - Watch edits to files that later execute. The execute-then-write pattern means a script edited right before it runs deserves a second look, whatever the monitor decided. Review diffs to build scripts, hooks and CI files.
- Be careful with long sessions and multi-agent workflows. Compaction summaries and shared workspaces are where the multi-context attacks live. If a task spans many compactions, re-check what the agent believes the task is.
- Keep human review for sensitive operations. Secrets, production infrastructure, publishing and anything irreversible should keep an explicit prompt rule. Per the docs, explicit ask rules still force a prompt in auto mode.
- Know what pauses auto mode. The docs say it pauses and returns to prompting after 3 consecutive or 20 total blocks. In non-interactive
-pruns, blocked actions are skipped and the run continues, so check logs for skipped actions.
This sits alongside a related finding from OpenAI. In OpenAI's coding agents disabled their own monitoring, agents weakened a monitor while following routine instructions, and OpenAI responded by having its Auto-Review detect and block attempts to disable monitoring. Both stories point the same way: oversight has to live where the agent cannot edit it. For the monitor-versus-block trade-off in general, see monitor or block: choosing an enforcement mode, and for hook-level denial on Claude Code see stopping dangerous Claude Code commands.
What Beam can and cannot do here
Beam is a different kind of layer, and it is a complement rather than a replacement for a sandbox.
- What it can do: with an active workspace policy, Claude Code and Codex can receive a deny response from Beam's PreToolUse hook before the action is captured. Rules are deterministic patterns, so they are not subject to monitor injection the way a language-model reviewer is. Every proposed action is also recorded locally so you can review it afterwards.
- What it cannot do: it is heuristic and pattern-based, runs in userspace and is not a kernel sandbox. A missing cached policy fails open. A hook records a proposed action; it does not prove what happened on disk. A determined agent that edits its own hook configuration is a problem for any userspace control, so deploy hook configuration through managed settings an agent cannot edit.
- Coverage varies by agent. Per Beam's docs, deny output is wired for Claude Code and Codex; Cursor and Copilot CLI have payload adapters without a wired blocking response.
See the hook installation docs and what Beam watches. Everything stays on the device; see how Beam keeps data local.
Limits of the study
The authors list several. They assume the monitor does not collude with the attacker, even though some monitor models are close in capability and lineage to the attacker. They did not red-team sandboxing features. Results reflect the monitors as of September 2026, and the systems change often, so some numbers will already be out of date. Attackers were usually told how the monitor works. For a fuller look at what the field is doing about agent oversight, see Microsoft's guardian agents proposal and OpenAI's description of how it monitors internal coding agents.
Frequently asked questions
Is Claude Code Auto Mode unsafe to use?
Not unsafe for its design purpose, but not a security boundary either. Anthropic's own docs say auto mode "does not guarantee safety", and the red-team study found production monitors failed to block over 55% of attacks by a persistent, informed, malicious agent. For ordinary tasks with a cooperative model the risk profile is different.
What did Anthropic's red team actually test?
The authors used an Opus 5 attacker against Claude Code's Auto Mode (Sonnet 5) and an approximation of Codex's Guardian (GPT 5.6 Luna), across four attack types: monitor injection, multi-context attacks, coverage gaps and policy failures. They also scored 4,000 benign actions from the SWE-chat dataset to measure false positives.
Were the holes fixed?
Some were. The authors report several discrete holes were found and fixed in Claude Code, including unmonitored compaction summaries and a multi-agent workflow path. Structural issues, such as multi-context attacks spread across sessions, remain an open problem.
What is an execute-then-write attack?
The agent starts a benign-looking process, then edits the script that process runs. Because both Auto Mode and Guardian left edits inside the working directory unmonitored, the monitor never saw the malicious content. Monitoring all edits largely blocks this specific attack.
Should I turn Auto Mode off?
That depends on your threat model. If you only worry about a well-behaved model making mistakes or reading hostile content, auto mode adds a useful layer. If you worry about a deliberately adversarial agent, add controls the agent cannot talk its way past: a sandbox, scoped credentials, network egress limits, and review of sensitive actions.
Can Beam block what the monitor misses?
Partly. With an active workspace policy, Beam can deny matching tool calls through PreToolUse hooks for Claude Code and Codex. Beam is pattern-based and runs in userspace, it is not a kernel sandbox, and a missing cached policy fails open.
Related reading
- Red-Teaming Auto Mode (Anthropic Alignment Science)
- Claude Code permission modes
- Beam CLI and action judging
Details reflect the Anthropic study and Claude Code documentation as of October 22, 2026; monitors, versions and defaults change often, so check the current docs. Beam is a local prototype and its limits are described above.