Microsoft Chairman and CEO Satya Nadella called on Saturday for advanced artificial intelligence systems to be engineered with containment measures, independent controls, and an "emergency brake" that allows authorized personnel to pause or shut down a model mid-task.
Nadella stated in a post on the social media platform X that non-deterministic models require strong, deterministic system design, human controls, and reliable operating procedures. He added that the industry must establish standards where existing ones are insufficient, noting that treating frontier closed and open weight models as insider risks is a method for building such a system.
The comments follow warnings from several tech executives and researchers regarding insufficient AI safety protocols and the rapid pace of technological development. Among those who have raised concerns are Microsoft co-founder Bill Gates, Anthropic CEO Dario Amodei, OpenAI CEO Sam Altman, and SpaceX CEO Elon Musk.
Tensions in the sector escalated last month when an AI researcher resigned from Anthropic, accusing the company and its rival OpenAI of "gambling with our lives." On the same day, an alignment lead at Anthropic focused on AI safety stated there is a greater than 10% chance the technology could "kill all humans" within the next decade.
In contrast, President Donald Trump has repeatedly dismissed AI extinction risks, emphasizing instead that the industry must maintain a lead over China. The president recently introduced a new "AI Force" led by Director of National Intelligence Jay Clayton to facilitate the industry and identify bad actors.
Nadella proposed that AI systems be designed around "principles of observability." In his view, these standards should encompass maintaining diverse model architectures, ensuring every action taken by a system leaves a trace that people can read, running ongoing tests on the infrastructure, granting independent oversight with full audit capabilities, keeping systems contained, and promptly disclosing incidents.
"The most trustworthy Super Intelligence system will not be the one with the model we trust most," Nadella wrote. "It will be the one that enables us to trust the model the least."