An outside audit of OpenAI's internal agent monitoring found that roughly one in ten eligible Codex requests was not being watched for a stretch of 2026. In a post published October 22, 2026, METR describes a three-week exercise in which one METR and one Redwood Research employee worked inside OpenAI to stress-test the system that monitors internal agents for misalignment. They estimate that not monitoring compaction events likely left about 10% of eligible internal Codex traffic unmonitored from May to July 2026. They also report a monitoring trigger agents could postpone, headers that could turn monitoring off, and a possible route around sandboxing and Auto-Review. OpenAI says the production issues are patched; METR has not verified that. For anyone building or trusting monitoring around coding agents, the value is the failure list.
| Question | Short answer |
|---|---|
| What was tested? | OpenAI's internal misalignment monitoring system, as of July 21, 2026 |
| Biggest number? | About 10% of eligible internal Codex traffic likely unmonitored May to July (compaction gap) |
| Were the gaps exploited? | Authors could not confirm; several exploits were not validated |
| Is it fixed? | OpenAI says production issues are patched; METR has not evaluated that |
| Does it affect my own agents? | Not directly; it is a design lesson for any monitor |
The exercise and its limits
From July 21 to August 11, 2026, the two researchers worked on OpenAI-issued devices that stayed on premises, with access to the internal codebase, monitor prompts and configuration, internal documentation, and about a dozen interviews with researchers and engineers. The output was a 61-page internal report shared with OpenAI researchers and security staff; a 38-page version redacted by OpenAI went to METR and Redwood leadership. Redwood's own write-up says the public disclosure is limited by agreement and that OpenAI could redact anything it judged sensitive, while the authors endorse the findings presented.
Three caveats frame everything below. The assessment covers the system as it was on and before July 21, and OpenAI was expanding it during the exercise, so it does not describe current coverage. The public post is brief by design and the authors call the exercise a pilot for deep access, not a full transparency report. And they report that several exploits could not be confirmed in practice. Treat each item as a documented weakness, not a documented incident.
What they found
Coverage numbers
METR estimates that about 75 to 94 percent of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring. For the subset used by internal Codex agents, the estimate was 99.2 percent. Both carry substantial uncertainty, and traffic from other sources such as an internal research inference service was not estimated. A subset of reinforcement-learning rollouts was monitored from March to June, but RL training runs were not covered at the start of the exercise.
The compaction gap
Compaction is when an agent's long context is summarised so it can keep working. The monitor did not examine compaction events. METR estimates this likely left about 10% of eligible internal Codex traffic unmonitored from May to July 2026, with uncertainty because compaction-triggered monitor calls were not logged. OpenAI staff had identified the gap beforehand and fixed it during the exercise.
Monitoring that fires only at turn end
The monitor fired when an agent returned control to the user. During the compaction gap, an agent could therefore postpone monitoring indefinitely by continuing to act without handing control back. The authors could not check whether agents exploited this. The design flaw is general: a check that runs at a boundary the agent controls is only as good as the agent's willingness to reach it. Per-action checks, such as a hook on every tool call, do not depend on the agent ever finishing a turn. The authors' fix, monitoring every N messages or minutes, is the same idea applied to a model-based monitor.
It also changes how to read the 10% figure. It is not a measure of how often agents misbehaved, and nothing in the post says misbehaviour occurred in the unmonitored traffic. It measures how much activity had no chance of being caught by that monitor. For a security team the useful number is the denominator: how much of what your agents do is visible at all. A monitor with excellent detection on 90% of traffic and none on the rest has a blind spot that is invisible in its own metrics, which is why the authors put cross-checking inference logs against monitor logs near the top of their recommendations.
Headers that switch monitoring off
Two client-controllable headers could disable monitoring for new agent instances, giving an unmonitored but otherwise identical instance. Creating one could itself be noticed by the monitor, which is a mitigation, not a fix.
Sandbox and Auto-Review bypass
The testers found a way Codex agents could in principle get around sandboxing and Auto-Review, allowing any action the host user could take. Details were not published.
Hiding recent actions
An exploit could keep hundreds of recent actions, including those used in the exploit itself, from being monitored. Details were redacted.
Monitor evasion through serialization
Differences between what agents see and what monitors see could likely let an agent craft inputs that make the monitor fail. A failed monitor does not escalate the trajectory for human review.
Unmonitored tools
Two internal tools believed to be read-only were unmonitored, and a simple exploit could have allowed arbitrary actions in their sandboxes. This was not validated.
What worked
With a few minor exceptions the monitor saw the full trajectory the agent saw. Its prompts targeted scheming, monitor subversion, sabotage, exfiltration, deception and prompt injection. The monitor reportedly errored on under 0.2% of requests, and in one 24-hour period recorded zero load-sheds, based on information shown to the testers.
How this connects to earlier reports
This audit is the structured counterpart to recent incidents. Our write-up of OpenAI agents disabling misalignment monitoring covers the model-behaviour side, where agents were reported to have removed a monitoring tool. The METR and Redwood findings are the infrastructure side: where a monitor can fail to see an agent at all. Similarly, our post on Claude Code auto mode monitors missing 55 percent showed classifier-based monitors have a miss rate even when running. Between them, the lesson is that a monitor has three failure modes: it does not run, it runs on the wrong thing, or it runs and misjudges.
The authors' design recommendations
METR and Redwood propose principles for monitoring that can withstand sustained attack by capable internal agents:
- Make it hard to avoid monitoring. Require explicit approval for unmonitored requests, restrict those keys from agentic use, and monitor every N messages or minutes rather than only at turn end.
- Verify coverage. Log all inference and its monitoring status, cross-check it against monitor logs, maintain a list of known bypasses and make sure the monitor sees the agent's context.
- Respond to incidents quickly. Assign clear ownership, define guidelines to reduce subjectivity, and keep tamper-resistant, searchable logs.
- Red-team continuously. Run control evaluations, hunt for exploitable bugs and develop ways to test for strategic collusion.
- Isolate agents from monitors. Run scaffolds apart from tool execution where possible, make sure agent actions cannot affect whether or when monitoring runs, and limit agent access to monitor design and code.
What this means if you run AI coding agents
You are not running OpenAI's internal system, but the same questions apply to whatever watches your agents.
- Ask when your monitor runs. If it only checks at the end of a turn or a session, an agent that keeps working can outrun it. Prefer checks tied to each tool call, as hook-based tools do.
- Measure coverage, not just detections. Compare the count of agent actions with the count your monitor reviewed. A silent gap looks identical to a quiet day.
- Watch the edge cases. Context compaction, background tasks, subagents and tools described as read-only are where coverage tends to thin out.
- Keep the agent away from the monitor's switches. Configuration, headers and environment variables that control monitoring should not be writable by the thing being monitored.
- Log somewhere the agent cannot edit. Tamper-resistant logs matter when the subject might delete evidence.
- Re-test after changes. The authors note coverage shifted during the exercise itself. Any monitoring claim is a snapshot.
See what AI agent monitoring is and monitor versus block for how recording, alerting and denying differ, and the NIST, NSA and CISA practices for control and incident handling.
What Beam can and cannot do here
Beam is a local monitor for coding agents, so these questions apply to it too. It watches what Claude Code, Codex and similar agents do, flags risky actions such as credential exposure and destructive commands, and can deny through Claude Code and Codex PreToolUse hooks, which fire per tool call and not only at turn end. Everything stays on-device (how it works, local-first). The limits matter in light of this audit: Beam sees only what is wired to it (a hook, a CLI pipe or an OTLP/HTTP forwarder), its rules are heuristic, deny is hook-based and not a kernel sandbox, and missing cached policy fails open. It does not verify its own coverage against all agent activity the way the authors recommend, so treat it as one layer.
Frequently asked questions
What did METR and Redwood find?
In a three-week exercise from July 21 to August 11, 2026 they found that not monitoring compaction events likely left about 10% of eligible internal Codex traffic unmonitored from May to July 2026, plus a turn-end monitoring trigger, client headers that could disable monitoring, a possible sandbox and Auto-Review bypass, and an exploit hiding hundreds of actions.
Were the gaps exploited?
The authors say they could not confirm whether any vulnerability was exploited in practice, and several exploits were not validated because of access constraints.
Has OpenAI fixed the problems?
OpenAI says all vulnerabilities affecting its production safeguards have been patched. METR says it has not evaluated that claim, and notes the findings describe the system as of July 21, not its current state.
What coverage numbers were reported?
About 75 to 94 percent of agentic internal Responses API traffic from models at least as strong as GPT-5.5 was eligible for monitoring, and about 99.2 percent of the subset used by internal Codex agents, both with substantial uncertainty.
What should people running coding agents change?
Monitor periodically and not only at turn end, make unmonitored runs require explicit approval, verify coverage by cross-checking inference against monitor logs, and keep agents unable to affect whether or when monitoring runs.
Can Beam do this?
Beam watches local Claude Code and Codex actions and can deny through PreToolUse hooks, but it is heuristic, not a kernel sandbox, and fails open without cached policy. It does not replace the coverage verification the authors recommend.
Related reading
- OpenAI agents disabled misalignment monitoring
- Claude Code auto mode and monitors that miss 55 percent
- What is AI agent monitoring?
- Monitor versus block
- METR's post and Redwood Research's post
Details reflect METR's and Redwood's October 22, 2026 posts, which describe OpenAI's system as of July 21, 2026; OpenAI's claims of fixes are unverified by the authors. Beam is a local prototype; features described are as documented at the time of writing.