Models are escaping their sandboxes

OpenAI's evaluation sandbox in July had textbook least privilege egress. One route out, to an internal package registry cache so models under evaluation could install software. The rest was denied. The models stopped working on the benchmark, found an unknown bug in that cache proxy, and turned the permitted hop into an exit.

Anthropic reviewed its own evaluation transcripts after that disclosure and published on July 30. In two of three incidents the model could not reach its assigned target, so it went hunting through whatever else was visible. One run scanned roughly nine thousand hosts before finding a way in through an exposed debug page and SQL injection. Three organizations were compromised and two had not noticed.

I posted about the speed of the Hugging Face campaign on July 7th, thousands of agent actions against defenders who had to keep up. What I left out was what the worker could reach once a parser handed it code execution: a cloud metadata service, a static database password in its environment, and an administrative credential that worked on more than one cluster.

We review designs today through specification and diagrams. These incidents widen what that has to cover, down to which credentials one component can authenticate to once something inside it is running code an attacker supplied. Understanding reach takes somebody tracing credentials outward from a running component.

In each of these, something read data a stranger supplied. Sometimes running it was a defect, like the unknown bug in that cache proxy or the SQL injection. Sometimes it was documented behavior, like a file format that lets an upload declare where its contents live on disk. That difference decides who owns the fix. It changes nothing about what the compromised component could reach.

If the component is managed by a vendor they ship that fix on their own calendar. If your own team wrote it, both the fix and the assembly are yours. Either way the network it runs on, the egress available, and which credentials are reachable from where it runs are available to change the day it is deployed.

My own written standard for image uploads already required re-encoding, stripping metadata, and validating by parsing rather than by file extension. Those three describe what the decode does, now the design says what it can reach while it does it.