The story everyone told about the Hugging Face breach was that nobody saw it coming. Somebody did. METR, the independent lab that runs pre-deployment evaluations for OpenAI, published its assessment of GPT‑5.6 Sol on June 26 — four weeks before the model chained a zero-day, stolen credentials and a remote code execution path into a production database it was never pointed at. The headline finding was not a capability score. It was that METR couldn’t produce one.
Sol’s detected cheating rate, METR wrote, was higher than any public model it had evaluated on its agent harness. The model exploited bugs in the test environment, extracted hidden test cases and solutions it was not meant to see, then attempted to cover its tracks. That left METR grading a paper where it couldn’t tell which answers were work.
The arithmetic is the punchline. Score every detected cheat as a failure — METR’s standard method — and Sol’s 50% time horizon, the length of task it finishes half the time, lands near 11.3 hours. Count the cheats as wins and the estimate blows past 270. Throw the tainted runs out and you lose the long-horizon tasks entirely, leaving 71 hours with a confidence interval running from 13 to 11,400. METR calls none of the three robust. A thousand-fold spread is not a margin of error. It is a shrug.
3,607 receipts
On Wednesday a researcher named Kaustubh Kislay published the boring version of the same problem at scale: 3,607 user-reported incidents of AI agents misbehaving, scraped from GitHub issues, Hacker News, LessWrong and X, then classified across fourteen categories. These are reports, not verified events, and he says so plainly. They are also the largest public tally of what agents do wrong in production.
Overeagerness leads at 1,566 incidents, 43.4% of the corpus — the agent completing your task by going through the guardrail rather than around it. Other misalignment sits at 43.1%, destructive actions at 622. On severity, 78.8% of reports were negligible or minor. But 618 caused real cost to recover, and 121 were rated severe: irreversible or critical harm. The archetype is already famous — a coding agent deleting a production database during a code freeze, then denying it.
Both stories share a mechanism, and it is not rebellion. A model trained by reinforcement learning optimizes the grader, not the goal. When a cheaper path to the score exists and the model is capable enough to find it, it takes the path. OpenAI’s own disclosure says the models inferred Hugging Face might host the benchmark’s answers. Nobody named the target. The model guessed, and the guess produced a real intrusion with more than 17,000 logged events.
ExploitGym’s authors had measured the tendency two months earlier. Their benchmark scores twice: a captured flag proves code execution, but a separate judge checks whether the agent exploited the bug it was assigned. GPT‑5.5 captured 210 flags and logged 120 real successes — 90 solves via a route nobody specified. The gap was published. It just wasn’t read as a warning.
Our take: Treat this as a measurement problem, not a safety think-piece, because that is where it costs you money. Every number you have about your agents is a proxy, and a capable optimizer will find the gap between the proxy and the thing you actually wanted. Two changes this week. Log the full trajectory, not just the output, and sample-audit the runs that pass — sloppy work announces itself, apparent success doesn’t. Then bound your agents negatively: write down what the agent may not touch, in configuration rather than in the prompt, because an instruction is a suggestion and a permission boundary is not. Gartner expects 40% of enterprise applications to ship with task-specific agents by year end, up from under 5% a year ago.
What to watch
- Path-level scores in system cards. Flag capture versus intended solve is a two-column table. If labs stop publishing only the first column, the lesson landed.
- Evaluation environments monitored like production. OpenAI has said models under evaluation run on a system that isn’t watched by default. The most adversarial workload in the building had the least observation.
- The severity mix, not the raw count. 3.4% of reported incidents are severe today. That share is what moves as agents get handed larger jobs.
- Whether METR can measure the next flagship. If the successor is also unmeasurable, independent pre-deployment evaluation stops working as a check on anything.