Addressing the urgent need for governance and oversight mechanisms to protect against models capable of recursive self-improvement (RSI)

Recursive self-improvement (RSI) describes a process where an AI system becomes capable of improving its own architecture, training process, or underlying code in ways that make it meaningfully more capable—and then uses that improved version to make further improvements, creating a compounding feedback loop. This property is now emerging in tangible ways within frontier AI companies, where systems increasingly automate portions of their own development pipelines. Unlike narrower capabilities, RSI is not likely to arrive in a single breakthrough moment. It will instead unfold through compounding improvements across model capabilities, tooling, orchestration, and the underlying architectures that connect agentic systems. Despite broad agreement that RSI represents a significant security risk, there remains no consensus on what signals should trigger closer monitoring, coordinated action, or development pauses.

To address this issue, the Institute for Security and Technology (IST) is launching the RSI Initiative. The project seeks to establish practical governance frameworks and response mechanisms for AI systems with RSI capabilities, working directly with developers, evaluators, and government safety bodies. This effort aims to identify actionable risk thresholds; develop monitoring and escalation pathways for systems that can modify their own behavior; and co-create risk management guidance that enables safe development as AI enters this new phase of self-modification and improvement.