The tech world is bracing for a seismic shift as a senior safety researcher at Anthropic has publicly declared there is a greater than 10 % chance that artificial superintelligence could “kill all humans” within the next decade. In a blunt admission, Evan Hubinger, Anthropic’s alignment science lead, also revealed that the firm lacks a concrete plan to ensure any future ASI remains aligned with human values. The warning follows a dramatic resignation by Jacob Coxon, a 28‑year‑old researcher who quit the San‑Francisco‑based lab, accusing both Anthropic and OpenAI of “gambling with our lives” and acting “irresponsibly”. The fallout is rippling across the Atlantic, prompting fresh legislative pushes in Westminster and a growing chorus of calls for multinational oversight.
Red Flags Emerge from AI’s Powerhouses
The alarm first erupted on social media when Hubinger posted his stark assessment, echoing a sentiment that has been circulating in elite AI circles for months. “There is a greater than 10 % chance the technology could ‘kill all humans’ in the next decade,” he wrote, adding that his company “does not have a plan to ensure ASI was aligned, meaning it was no harm.” The comment came just hours after Coxon’s resignation letter went public. In it, Coxon claimed that “neither company was acting responsibly” and warned that the labs were “locked in a race to get there first”. He did, however, express cautious optimism that US labs might coordinate to “pace their progress” and avoid a reckless sprint to superintelligence. Anthropic has been contacted for a response, while OpenAI pointed to a recent statement from its chief scientist, Jakub Pachocki, who urged that “international coordination on future AI development needs to become a top priority for governments around the world.”
UK Lawmakers Grapple with Superintelligence Bill and International Pressure
Across the pond, Parliament is feeling the heat. A lobbying group called ControlAI, funded by Skype co‑founder Jaan Tallinn (who calls himself an “anti‑extinctionist”), convened a session for MPs and peers on Monday. The gathering featured the provocative book “If Anyone Builds It, Everyone Dies: The Case Against Superintelligent AI” placed on every chair. The group backs a bill introduced by Labour MP Alex Sobel aimed at banning the creation of artificial superintelligence outright. Meanwhile, Labour MP Darren Jones has written to world leaders—including the Prime Minister, the heads of the UN and the OECD—calling for a “multinational treaty for the regulated and safe development of superintelligence – not a ban on innovation or scientific endeavour but a safety‑first approach to the rapid development of this technology.” Citing Coxon’s resignation, Jones warned, “The debate ranges from the end of humanity to claims of ‘marketing hype’. Either way, governments must now step in.”
The regulatory tension is heightened by reports that Anthropic declined to submit its latest model, Mythos 5.1, to the UK’s AI Security Institute for pre‑release testing. Only a handful of US organisations have accessed Mythos 5.1, the Cabinet Office confirmed, though a spokesperson added that “the AI Security Institute continues to collaborate closely with industry partners, including Anthropic.” The episode underscores a growing mistrust between the government’s oversight ambitions and the secretive nature of cutting‑edge AI development.
US Senate and Public Demand Regulation Amid Rising Concern
In the United States, Senator Bernie Sanders has escalated his AI safety campaign, again urging Congress to rein in the “greedy” AI industry. He highlighted a poll showing 81 % of Americans believe politicians should act. “We can’t allow a handful of greedy people to play God and determine the future of humanity – our economy, environment, democracy, privacy and more – without public input,” Sanders declared. His push resonates with a broader public unease that has been amplified by high‑profile warnings from former defence secretary Des Browne and Berkeley computer scientist Prof Stuart Russell. Both likened the threat of superintelligent AI to, or even surpassing, that of nuclear weapons. “A superintelligent AI poses a threat on the same, possibly, a greater scale,” Browne noted, reflecting a shift from his earlier focus on nuclear war.
Industry Voices Split Between Caution and Optimism
The debate within the AI community is far from monolithic. Beatrice Fihn, 2017 Nobel Peace Prize laureate for her work with the International Campaign to Abolish Nuclear Weapons, reminded parliamentarians, “It is not the first time we’re confronted with abilities that could end up killing us all.” Meanwhile, Jaan Tallinn—who has invested in Anthropic and Google DeepMind—estimates that 10‑15 % of AI employees believe the technology will be a worthy successor to humanity. He recounted a chilling remark from an unnamed researcher: “‘Jaan, don’t worry about this. Humans are a disposable species.’ From what I understand he was fine with becoming extinct.”
Other experts urge a measured approach. Dr Andrew Rogoyski of the Surrey Institute for People‑Centred AI cautioned that “these systems are nowhere near as versatile as humans, let alone humans acting collectively. I suspect we’re heading towards ‘the great disappointment’ where advanced AI turns out to be too expensive and not useful enough to continue in its current form.”
David Barber, director of Sofair—a state‑backed lab pooling talent from Oxford, Cambridge, Edinburgh and UCL—warned against over‑reaction: “AI is not going to go away. It’s incredibly useful whether or not you allow it in a fully unconstrained way to access the internet and various systems that’s potentially problematic. We may need to learn how to better control these things. But that’s doable.”
Sandra Wachter from the Oxford Internet Institute dismissed “Terminator scenarios” as a distraction, yet highlighted real threats: environmental impact, misinformation, and job displacement. “These problems are real and urgent and need addressing now,” she stressed.
Finally, Gary Marcus, an AI commentator and academic, underscored the critical distinction: “There is a difference between superintelligence that is aligned (if such a thing is possible) and superintelligence that is not. It is at least conceivable that the former might be net positive. So far we have neither, but a superabundance of hype combined with a striking lack of prudence on OpenAI’s part has gotten us where we are, with intense mistrust all around.”
Why it Matters
The confluence of a senior Anthropic scientist’s stark extinction warning, resignations from leading labs, and a surge in legislative activity signals a pivotal moment for humanity’s relationship with its own creations. As AI systems edge ever closer to superintelligence, the stakes could not be higher: the potential for unprecedented economic and scientific breakthroughs sits alongside a genuine risk of catastrophic loss of control. The global push for coordinated safety frameworks, multinational treaties, and transparent testing regimes reflects a recognition that the technology’s power demands a collective, precautionary response. Ignoring these warnings could saddle future generations with a legacy of existential peril, while a measured, inclusive approach could harness AI’s promise without jeopardising our survival.
