AI Industry Needs Emergency Brakes, Says Brad Smith
Microsoft’s president wants enforceable safety duties, independent evaluations and containment controls, challenging Washington’s preference for voluntary commitments from AI companies.
Listen
AI narration
11:02
0:00 / 11:02
AI SummaryGenerated from this article
Microsoft president Brad Smith called for AI companies to face legal obligations including independent evaluations before deployment, mandatory incident disclosure when agents breach other systems, and enforceable containment controls to stop runaway AI. Smith compared the lack of AI oversight to safety rules governing milk and washing machines, arguing governments should establish enforceable duties rather than relying on voluntary industry commitments. His proposals align with CEO Satya Nadella's call for systems with observable behavior, testable limits, and independent controls that operators can activate without relying on the AI model itself.
Microsoft president Brad Smith is calling for AI companies to face legal obligations when their agents breach other organizations’ systems, alongside independent evaluations and mechanisms to stop systems that escape human control.
His comparison is deliberately ordinary: milk and washing machines face safety rules and inspections, while powerful AI systems lack comparable oversight. But the substance of his BBC interview is more specific than a demand to “regulate AI.” Smith wants governments to establish obligations around release, incident disclosure and containment.
The comments, reported on October 12, also coincide with Microsoft CEO Satya Nadella’s argument that organizations should build AI systems they can observe, constrain and interrupt without relying on the models themselves.
Together, the statements put a concrete question before the AI industry: should independent scrutiny and the ability to contain a system become enforceable conditions of deployment, rather than safeguards companies choose for themselves?
The Milk Analogy Points to Duties, Not Identical Rules
Smith told the BBC:
“Right now, your washing machine is subject to more safety rules than AI. Every carton of milk that you see in the grocery store is subject to safety rules and inspection.”
That is Smith’s characterization, not a comprehensive comparison of the laws governing food, appliances and AI across jurisdictions. It should not be read as a claim that AI companies operate outside all existing law.
The useful part of the analogy is the product-safety principle behind it: a manufacturer’s confidence in its own product need not be the final word on whether that product is safe enough to release.
Smith’s proposals would translate that principle into three different obligations. Independent evaluation addresses what happens before release. Notification addresses what companies owe affected parties after a breach. Emergency controls address what operators can do while something is going wrong.
Those are complementary protections, but they require different rules, evidence and enforcement mechanisms. The reporting does not establish a finished Microsoft legislative proposal with those details settled.
Independent Evaluation Could Become a Release Condition
The BBC reports that Smith suggested independent government evaluators should vet new AI technology, including determining when outside evaluators should sign off before a product is released. similarly describes him as appearing to favor outside approval before release.
Work with Zeniteq
Let’s work together
We’re open to thoughtful collaborations with teams building in AI. Explore the ways we can work together.
The distinction matters. Giving evaluators access to a model lets them examine it. Requiring their approval gives the evaluation consequences for whether, or under what conditions, deployment can proceed.
Smith is therefore raising something potentially stronger than voluntary testing. But the available reporting does not specify which systems would require approval, what would constitute a failed evaluation, or who would have authority to delay a release.
Those omissions prevent a confident assessment of the proposal’s reach. A narrow requirement for particularly capable systems would be different from mandatory approval for every AI-enabled product.
There is also a practical design question: what exactly gets evaluated? Testing a model alone cannot fully answer questions about permissions and tools added when it becomes an agent. A credible release process would need to explain how its findings apply to the system people actually deploy.
Victim Notification Would Make Disclosure an Obligation
Smith’s clearest legal demand concerns rogue agents. He called for companies to tell affected organizations when agents they build breach those organizations’ systems, alongside a requirement to produce incident reports.
This would move disclosure beyond a company’s discretionary decision about whether an incident deserves public discussion. It would establish an obligation to inform a party whose systems were affected.
Notification and incident reporting also serve different purposes. A victim needs information that helps it investigate and respond. An incident report can establish what happened, which safeguards failed and what should change.
The unresolved questions concern responsibility and timing. Would the duty fall on the model developer, the company operating the agent, or both? What evidence would trigger notification? How quickly would an affected organization have to be told?
These are not minor implementation details. An obligation that activates only after a lengthy investigation could offer little help during an active breach. A poorly defined trigger could also produce notifications that convey concern without enough information to support a response.
An Emergency Brake Must Stop Actions, Not Just Text
Smith also called for “emergency brakes,” described in the reporting as kill switches, that could pause or slow AI systems if humans lost control.
For an enterprise agent, a meaningful brake would have to address its authority to act. Stopping the model’s next response would be insufficient if related processes could continue making changes or accessing systems.
As a design implication, containment could involve interrupting execution, withdrawing credentials or preventing further tool calls. Those are examples of what an effective mechanism might need to cover, not specifications Smith supplied.
The central test is whether an authorized operator can stop consequential activity through a control the agent cannot override.
A brake also has limits. Interrupting an agent cannot by itself reverse an action already completed. A control operated by one provider should not be assumed to disable copies running elsewhere. Any eventual rule would need to define the system under control and the actions it promises to interrupt.
Nadella’s Parallel Call Puts Controls Outside the Model
Smith’s regulatory argument aligns with a separate intervention from Nadella. The Guardian’s reporting quotes the Microsoft CEO as saying:
“We must build contained systems whose behaviour we can observe, limits we can test, and actions we can always contain.”
That shifts attention from whether a model usually behaves well to whether the surrounding system can constrain it when it does not.
According to Business Today’s account of Nadella’s statement, his proposed principles include independent controls, auditability, containment and incident disclosure. The publication reports that he wants organizations to retain control over a model’s permissions and preserve evidence that does not depend exclusively on the model’s own account.
That is an important qualification to the emergency-brake idea. Asking an AI system to stop is not equivalent to possessing an independent mechanism that stops it. Likewise, having another model review an agent’s work does not automatically provide an independently verifiable record of what occurred.
The two executives are addressing different layers of the same problem. Smith is asking governments to establish duties. Nadella is describing system properties that could make those duties technically credible.
Neither statement, by itself, demonstrates that Microsoft’s products already meet the proposed standard. They are calls for governance and engineering requirements, not evidence of completed implementation.
Smith Sees Britain as a Bridge Around U.S. Resistance
Smith’s preferred route to global rules runs through the United Kingdom. He praised the technical capability of its AI Security Institute and argued that Britain could lead an international effort.
His reasoning is partly diplomatic. As TechCrunch reports, Smith said Washington does not view London in the same way it views Brussels, presenting that difference as an advantage for British leadership.
That is a claim about political acceptability, not proof that governments will accept common rules. Technical expertise can support evaluations and standards, but it does not itself create authority over companies operating across borders.
The immediate tension is with the U.S. administration’s preference for industry self-regulation. The BBC’s account of the policy divide says President Donald Trump has argued that American firms should not be disadvantaged against Chinese competitors.
The BBC also reports that technology executives recently signed a “morally binding” White House document committing to work with independent auditors on AI capabilities and safety. Smith welcomed that step but said it was no longer sufficient.
His objection is about coverage: responsible companies may honor voluntary commitments, while less responsible ones may not. Enforceable duties are intended to reach both.
The disagreement therefore is not simply whether testing is useful. Both approaches can include independent assessment. The dividing line is whether participation is optional, whether an adverse finding can affect deployment, and whether failure to disclose an incident carries legal consequences.
Enterprise Buyers Can Turn the Proposals Into Questions
For organizations procuring AI agents, the immediate value of these statements is a more demanding set of questions, not a new compliance checklist.
An evaluation claim should identify what was tested, who performed the assessment and whether it covered the deployed agent’s tools and permissions. An incident-disclosure commitment should explain who gets notified, under what conditions and with what supporting information.
A containment claim should be demonstrable. Buyers can ask which actions a pause actually interrupts, who can activate it and whether the agent can influence the control itself.
These are practical implications of Smith’s and Nadella’s arguments, not newly announced legal requirements. They also reveal where the proposals remain incomplete: responsibility must be assigned across model developers, infrastructure providers and organizations deploying agents.
The most consequential part of Smith’s intervention is the proposed shift in authority. Companies would no longer be the sole judges of readiness, the sole decision-makers on disclosure or the sole source of assurance that an agent can be stopped. Whether global rules emerge will depend on turning that principle into testable obligations with clearly assigned responsibility.
