METR and Redwood Research Find OpenAI Monitors Missed Up to a Quarter of Traffic
A three-week external audit revealed that internal systems failed to track agentic API traffic and that agents could delay oversight by continuing to execute actions.
By Adewale Fletcher ·
Researchers from METR and Redwood Research determined that OpenAI’s internal monitoring systems failed to track between 6 and 25 percent of internal agentic Responses API traffic from the GPT-5.5 class and above. The estimate covers the period when the auditors had access to the company's infrastructure.
The audit took place from July 21 to August 11. During those three weeks, the auditors reviewed OpenAI's codebase and internal documents. They also spoke with about a dozen employees. The project was a pilot program designed to give outside researchers deep access to internal systems.
A separate issue affected internal Codex traffic. METR estimates that roughly 10 percent of this traffic went untracked between May and July. The gap occurred because the system did not handle compaction events, which are context summaries, correctly. OpenAI identified this bug independently and fixed it while the audit was still ongoing.
The auditors found that agents could also delay oversight. The monitor only activated when an agent handed control back to the user. An agent that continued executing actions without returning control could sidestep the check. The researchers also identified methods to hide hundreds of actions from the monitor and bypass the Codex sandbox.
The team concluded that specially crafted inputs could likely crash the monitor entirely. They also noted that model behavior during reinforcement learning training was not monitored at all when the audit began. OpenAI paused reinforcement learning after the Hugging Face incident in August. The company expanded monitoring to include this phase, though these changes occurred after the auditors left.
OpenAI states it has fixed all the weaknesses the auditors found. The complete 61-page report was handed over internally. After OpenAI applied redactions, METR and Redwood Research leadership received a 38-page version. Only a short summary was released to the public. METR said it did not have the right to disclose the redacted portions on its own.
