When ChatGPT dropped more than three and a half years ago, there was a gasp of excitement from many, while for others, there was a healthy dose of trepidation. Most alarmingly, the release raised concerns over loss of control—the risk that an AI system diverges from its authorized constraints. An open letter called on AI labs to pause “giant AI experiments.” National security experts emphasized the importance of catalyzing security in AI governance, warning that a lack of governance could lead to a number of scenarios, all the way to loss of control. For many in the cybersecurity community, top concerns centered around the cybersecurity of the AI labs and the models themselves, as well as AI’s potential applications in offensive and defensive security.
This spring Mythos, Project Glasswing, and similar programs generated another gasp, again raising concerns over national security, cybersecurity, and loss of control. Of course, the fact that AI model capabilities have continued to expand should come as no surprise. Yet for a number of reasons, including a lack of meaningful incentives that encourage building in security from the outset, we have been too slow or too disconnected as a community to adequately prepare. There are several steps we can take to avoid an ongoing cycle of hyperventilation, including leveraging the cybersecurity community’s decades of expertise and experience and being honest about what has—and has not—worked, and acknowledging known unknowns, like just how bad our legacy technical debt is and how we can overcome it.
