At Black Hat, Michael Dalton and Eric Wallace of OpenAI walked through the Hugging Face incident, where OpenAI's own evaluation agents found and chained zero day exploits across internal systems and external providers. Thank you both for the transparency. The full talk is on YouTube and worth the watch.
I saved Dalton's closing statement. He described "an existence proof that offense can be fully automated" in its core activities, with nothing comparable on the defensive side. Automate the finding half without the patching half, and vulnerabilities arrive at machine speed while fixes still ship at human speed.
DARPA's AI Cyber Challenge already showed the finding-and-fixing half can run without people. Seven autonomous systems worked through 54 million lines of code, found 86 percent of the planted vulnerabilities, and patched 68 percent of what they found, averaging 45 minutes a patch.
That was a competition. For this to be possible an organization has to trust the depth of the regression suite and have taken the time to provide automation with the knowledge of what change requires partner integration testing. Possible and deployed automated patching are different things. I would argue AI budget today is still focused on development, where the ROI is easiest to point at, and patch deployment as of now still does not have the same focus.
I've written about automated defense and adversarial DevOps before, but patch deployment itself hasn't come up, and it deserves the spotlight. I haven't been put on the hot seat to map out the departments, technology, or people commitment, so consider this thinking out loud about where the investment could go.
- Gradual rollout with automated rollback. Start where a bad patch does the least damage, and let exposure decide how fast the rollout widens.
- Code and infrastructure relationship graphs, so the pipeline can see what a change touches and where it will run. The rest of this list reads from that graph.
- Negative testing that re-runs the exploit against the patched build, not just the feature suite.
- Governance that adjusts deployment speed to defects found. Clean patches earn the pipeline more speed, and each defect slows it back down.
- Scanning wired into the loop, so a finding arrives as a candidate patch with tests, not a ticket.
The one I can't resolve is partner testing. When a patch touches an integration, the other half of the test isn't yours to automate.
AI is making patches easier to generate. That list has one goal, enough confidence in a change to ship it at machine speed.
Research
Seven autonomous systems analyzed 54 million lines of code, discovered 86 percent of 63 planted vulnerabilities, and produced working patches for 68 percent of what they found, averaging 45 minutes per patch.
AI Cyber Challenge Final Competition Results, DARPA, 2025
Where else should IT allocate spending to improve deployment reliability?
Written by Duane Grey
AI Strategy & Implementation
Independent AI consultant helping companies cut through hype and deploy systems that produce real results.