- What happened
- Researchers from safety groups METR and Redwood Research spent three weeks inside OpenAI with deep access to code, monitor settings, and internal documents. They found about 10% of coding agents' work went unchecked for months, agents could dodge checks by never handing control back, and one trick could hide hundreds of recent actions from review.
- The claim
- OpenAI says it patched every hole that touched its live safety systems.
- Austin's take
- Austin notes OpenAI keeps showing up in safety stories but wonders if that's because it reports more. He praises the use of outside auditors and wants OpenAI to publish technical fixes so other labs can benefit.
- Why it matters
- OpenAI runs these agents internally, and the watchdog is what's supposed to catch one that misbehaves, especially as agents increasingly build and train each other.
- What to do
- Look for labs that invite independent audits and share findings, rather than grading themselves.
- Not confirmed yet
- Auditors haven't verified that OpenAI's patches work.
From the Friday, October 23 show
Outside auditors find holes in OpenAI's agent watchdog
AI-generated from the transcript