OpenCode's AI refused to help it fight off an AI-driven attack

What happened: Attackers used AI agents to hit AI coding tool OpenCode. When the team fought back, the frontier models helped at first, then the attack code tripped their safety filters. The attack went on for 48 hours, and some users found up to $1,000 in charges they didn't make. OpenCode got access to less restricted models through OpenAI's Daybreak Blue program. Hugging Face hit the same problem in July. White House tech adviser David Sacks said the guardrails backfired.
The claim: Hugging Face said guardrails can't tell a responder from an attacker. Only vetted teams get access to the looser models.
Austin's take: Austin explains that labs keep stronger cybersecurity models gated to keep them from bad actors. He says OpenCode CEO Dax contacted Anthropic and OpenAI to get onto those models. He wonders if labs should proactively offer defensive models to companies (likely for a fee) and notes that attacks keep getting caught and stopped thanks to observability tools.
Why it matters: Defenders can't afford an AI that quits mid-fight while attackers move fast, which raises the question of who should decide access to unrestricted models.
What to do: If your team relies on AI for security response, look into vetted-access programs before an incident, not during one, and watch your accounts for unexpected charges.

The Loop