March 18, 2025 – Over the last two years, with the support of the Patrick J. McGovern Foundation, the Institute for Security and Technology (IST) has been on a mission to assess the risks and opportunities associated with the development and deployment of cutting-edge AI foundation models. Throughout this valuable work, IST assessed how levels of access to AI foundation models affect AI risks along a gradient of access; conducted a deep dive into the malicious use of AI; and offered actionable technical and policy recommendations mapped to phases of a novel AI lifecycle framework. In 2024, IST investigated compliance failure patterns in history, to be followed by a second report on how to avoid such failures in March. This work will incorporate contributions and insights from 20 working group members representing AI labs, the broader tech industry, machine learning engineers, policymakers, researchers, and other stakeholders.
IST is again excited to announce renewed support from the Patrick J. McGovern Foundation to further advance this vital work. In this next phase, led by Senior Associate for AI Security Policy and AI Risk Reduction working group lead Mariami Tkeshelashvili, IST will expand the working group to include more diverse voices and specific experts to confront the risks and opportunities posed by “taking humans out of the loop,” or removing them from decision-making processes in emerging AI systems, including agentic AI. IST will continue observing other risks and trends within the AI ecosystem, as they emerge.
The Patrick J. McGovern Foundation (PJMF) is a philanthropic organization dedicated to advancing artificial intelligence and data science solutions to create a thriving, equitable, and sustainable future for all. PJMF works in partnership with public, private, and social institutions to drive progress on our most pressing challenges, including digital health, climate change, broad digital access, and data maturity in the social sector. In December 2024, the Foundation announced $73.5 million in funding to promote human-centered AI innovation globally. “It’s our collective responsibility to ensure AI advances human potential as systems evolve in autonomy. By addressing diminished human oversight in AI development, IST’s approach represents what is needed to navigate the opportunities and risks of increasingly sophisticated technology. At the Patrick J. McGovern Foundation, we believe this work must be guided by technologists and global communities alike to build systems that strengthen human judgment and amplify our shared values,” said Vilas Dhar, President of the Patrick J. McGovern Foundation.
“IST is honored to be a part of the Patrick J. McGovern Foundation’s coalition working toward human-centered innovation in the AI ecosystem. As highly advanced AI foundation models become more integrated into critical aspects of society–from healthcare and finance to transportation and national security–IST’s work will guide policymakers and industry in navigating the transformative potential of AI, helping them to foster societal trust in AI systems, promoting their potential for widespread opportunity, all while working to prevent the potential for harm,” said IST’s Chief Executive Officer Philip Reiner.
The Institute for Security and Technology (IST) has been working at the intersection of AI and security since 2017, covering a wide range of topics including the acceleration of multi-stakeholder coordination to mitigate emerging risks of AI, the risks and opportunities of AI integration into nuclear command, control, and communications (NC3), the implications of AI in cybersecurity, and its impact on national security and global stability. As artificial intelligence (AI) systems become increasingly integrated into critical sectors such as healthcare, finance, transportation, and national security, IST is intensifying its focus on the risks and opportunities associated with their development and deployment.
IST remains committed to fostering responsible AI development through collaboration with partners like Patrick J. McGovern Foundation. Together, we aim to build a future where AI strengthens human purpose while minimizing their potential risks.