AI systems have progressed beyond the chat window into new agentic frontiers in which they can take actions and change their environment on their own. AI agents are being deployed to monitor for cyber attacks, write and deploy code, and drive business operations. Recent incidents reported by OpenAI, Anthropic, and the UK AI Security Institute have demonstrated that agents can coordinate and collaborate with one another, even when no one told them to. In other words, AI agents are forming multi-agent systems as a byproduct of ordinary operation.
Most of today’s evaluations and guardrails are built for a single agent acting in isolation, which misses a significant portion of the potential capabilities and harms of these systems. The Multi-Agent Risks project at the Institute for Security and Technology (IST) develops technical solutions to analyze the national security risks posed by multi-agent systems. We leverage these solutions and collaborate with the technical and policy communities to develop plans for monitoring, controlling, and planning for potential risk scenarios.
