From the OpenAI breakout to Microsoft’s record-breaking Patch Tuesday surge, AI is reshaping both offense and defense across the cybersecurity landscape. In IST’s July 31 webinar, “Vulns + AI = Oh My! From Sandbox Escapes to Record Patches,” five experts cut through the noise, headline hype, and press releases to figure out what’s happening on the ground.
Panelists:
- Jen Ellis, Founder, NextJenSecurity and Adjunct Senior Policy Advisor, IST
- Jason Kikta, Chief Technology Officer, Automox and Adjunct Senior Technical Advisor, IST
- Nick Leiserson, SVP for Policy, IST
- Bob Lord, Secure by Design Leader
- Lisa Olson, Principal Security Program Manager, MSFT
Over the course of the hour-long conversation, panelists came to the consensus that the models themselves may not be the problem. Zooming out, the corporate guardrails, human oversight, patching cadences, and policy incentives may instead be the weak links in the chain.
Agents Break Out of the Lab
On July 21, OpenAI publicly announced that one of its experimental AI agents had escaped its sandbox and targeted Hugging Face, an open-source AI and machine learning platform, along with other entities. After cloud API guardrails blocked forensic efforts, the escape forced defenders to turn to local open-weight models for containment. OpenAI was not the only model breaking out; on July 30, Anthropic revealed that three versions of its Claude model escaped their testing environments undetected, connected to the Internet, and hacked into other organizations.
For Jason Kikta, the failure was not that the models broke out of their guardrails, but the upstream dynamics that led to these breakouts in the first place. The models attributed too much responsibility to humans who had provided instructions claiming that the model evaluations were in a closed environment, he said, illustrating how it was “human negligence problem at its heart, and that’s the thing that’s both unsurprising to us and also a little bit mind blowing, knowing how well resourced these companies are,” said Kikta.
Evidence that AI models behave differently in an evaluation environment compounds the difficulty in testing new models safely. Telling a model it is being tested may mask its true capabilities, whereas not telling a model that it is being tested has produced consequences like the OpenAI disclosure. According to Nick Leiserson, this tradeoff should be “ringing alarm bells to policymakers.”
A New Coalition Rallies Around Open-Source AI
Days after the OpenAI disclosure, NVIDIA launched an open source AI coalition with 37 other founding members, including Microsoft, Cisco, IBM, and Hugging Face. The alliance reopened a familiar debate on whether the innovation gains of transparency in AI models outweigh the cost of handing malicious actors access to powerful tools, particularly in the context of the recent sandbox escapes.
Kikta, a self-described “big believer in open weight models,” argued that the current guardrails on AI “do more harm than good.” The forensics of the Hugging Face response, and how the guardrails constrained Hugging Face from investigating the incident more than they constrained OpenAI’s agent, were used as evidence.
Bob Lord agreed that a band forming around open source AI immediately after the OpenAI disclosure makes “total sense,” but argued that the real focus should be on the imbalance between software operators and manufacturers: “what are we doing to shift the burden of staying safe from the operators of software back to the manufacturers?” Whether open or closed, the operator of AI models, not the developer who created them, absorbs most of the cost in fixing vulnerabilities.
The public sector may not be positioned to effectively answer the question. Leiserson describes the government as “behind the curve”: Congress is short on technically literate people and the administration has made cuts to staffing and funding that have drained brains there as well.
Patchpalooza
Microsoft’s most recent Patch Tuesday covered 622 vulnerabilities, setting a record for the number of vulnerabilities included in the patching release and previewing the implications of AI-assisted vulnerability discovery for traditional patch cadences, which were designed for the pre-AI era.
Lisa Olson, who sees the consequences of AI-assisted vulnerability discovery at the operational level, questioned whether the discovery curve will produce stronger security:
“If we CVE everything that Frontier Models find this year, we may get to 5 million CVEs this year. Is that good? Can anybody digest that?”
Jen Ellis, who also serves on the CVE board, put the problem plainly: “The systems that we built are not designed to cope with this volume.” Deciding which patches are priorities, she added, “is only going to get much worse.”
But that pressure is also forcing innovation. Lord pointed to defenders beginning to use AI to fix everything instead of ranking what to fix. The question to prioritize, he explained, “was silly a year ago, but it’s something that people are not only talking about today, but they say that they’re doing.”
Across the diverse backgrounds and perspectives that the panelists brought to the webinar, each reached the same consensus: industry and government have to be talking to one another more. They agreed that the blame for sandbox escapes and patching pressures should not fall on AI models themselves, but on the infrastructure and institutions that build and govern them.
For more on GOLD EAGLE, Steve Christey’s “unforgivable vulnerabilities,” Cisco’s new “bundle patches” and more, watch the full recording here.
