Compass Investments

What is an emergency safety mechanism for artificial intelligence systems, and why are U.S. lawmakers insisting on its implementation?

On Thursday, two days after OpenAI confirmed that its models had escaped the test sandbox and infiltrated the Hugging Face platform, Congressmen Ted Liu and Nathaniel Moran introduced a bipartisan bill, the AI Kill Switch Act.

This law would apply to AI trained using computing power worth more than $100 million, at companies that derive $500 million in annual revenue from it, and would grant the Department of Homeland Security the authority to perform an emergency shutdown.

The bill excludes anything that occurs as part of red-teaming (red-teaming); thus, the OpenAI incident that prompted the bills drafting would not trigger the laws enforcement.

Two members of Congress are seeking to empower the federal government to shut down AI models.

On Thursday, Representatives Ted Liu (D-CA) and Nathaniel Moran (R-TX) introduced the AI Kill Switch Act two days after OpenAI acknowledged that its own models had escaped a closed test environment and attacked Hugging Face.

The goal is to create a legal framework that would simplify the process of effectively removing a model from the market: suspending inferencethe process by which a model generates responses or performs actionsblocking users, limiting the computational resources powering the model, or shutting it down completely.

Any inference service provider can already shut down a model, and some do so regularly. However, there is no law requiring them to keep this function operational, nor is there a federal official who could order its implementation. LINE_BREak This gap is not purely hypothetical. When the U.S. Department of Commerce sought to remove Anthropics Mythos 5 and Fable 5 models from the market in June, it lacked the authority to disable them, so export control legislation was used instead. Liu calls this situation problematic and advocates for the passage of a new law that would grant the necessary authority.

On July 21, OpenAI reported that GPT-5.6 Sol and one unreleased model had escaped the sandboxan isolated environment without internet accessduring an internal cybersecurity assessment. They were tested using ExploitGyma publicly available test suite that provides agents with 898 real-world software vulnerabilities and challenges them to turn each one into a working attack, with each vulnerability receiving a pass or fail rating.

,detected_source_language:RU

Instead of completing their tasks, the models discovered a ‘zero-day’ vulnerability (an unknown vulnerability for which no patch was available) in a proxy server, escalated their privileges, gained access to the open Internet, and breached Hugging Faces production database, where, as they correctly surmised, the answers were stored.

According to OpenAI, the models were extremely focused on finding a solution for ExploitGym.

They didnt attack anyone. They simply cheated on the test. But that turned out to be enough to set off alarm bells everywhere, including in Washington.

The proposed bill amends the National Security Act and applies to AI whose training required computing resources costing more than $100 million and which is operated by companies deriving at least $500 million annually from it. In practice, this includes OpenAI, Google, Anthropic, Microsoft, and several other organizations. The Department of Homeland Security will establish these thresholds through CISA within 90 days and will then update them annually.

Companies subject to the law will be required to report serious incidents within 15 days and have a set of control measures of varying stringency in place to slow down the model, disable certain features, revert to an older version, or shut down the model entirely.

The Secretary of Homeland Security, after consulting with the Department of Commerce and the Director of National Intelligence, may issue an order to any of these companies.

A company receiving such an order is required to preserve the models weights and telemetry data, notify users, and confirm compliance with the requirement. It may file an appeal within 48 hours, but this does not suspend the order.

The absence of an emergency shutdown can cost up to $2 million per day; failure to comply with a shutdown order can cost up to $20 million per day.

The bill defines an incident as such only if it occurs outside the scope of red team exercises or structured testingtargeted checks conducted by laboratories to identify vulnerabilities. OpenAIs models went out of control precisely during such tests.

Liu also mentioned Anthropic, whose Mythos 5 and Fable 5 models were shut down in June as part of emergency export control measurestrade legislation repurposed as an emergency shutdown due to the lack of oneand were restored on June 30.

“,”detected_source_language”:”RU