By Jessica Hamzelou

In recent weeks, leaders of some of the biggest AI companies have warned that the very tech they are developing is dangerous. Last weekend, Anthropic CEO Dario Amodei argued that AI carries serious risk and that progress should be slowed. OpenAI CEO Sam Altman responded on X: “I agree with Dario that we need to pace the frontier.”

Those posts came a few days after the AI researcher Jacob Coxon announced that he was leaving a role at Anthropic, charging that neither it nor OpenAI (where he had also worked) was acting responsibly. “The people building AI earnestly believe that it could kill us all by the end of the decade,” he posted on X. Another Anthropic employee, Evan Hubinger, publicly agreed with him. “We really do earnestly believe AI could kill all humans!” he responded on X. “I personally think it is >10% within the next decade.”

In a report published last week, Anthropic acknowledged that people had attempted to use its models to explore ways to make the chikungunya virus more transmissible, create a form of bird flu that is more dangerous to humans, and build an “atlas of venom toxin peptides,” among other things.

Kevin Esvelt, an MIT biologist who invented both technology to fast-track the propagation of a genetic feature through an entire population and ways to limit that technology, echoed these concerns in an X poston Wednesday, stating that a large language model had “disclosed a novel form of bioweapon that I hadn’t realized was possible.”

He added, “Please, for the love of God, children, the future of humanity, or whatever you consider holy, let's err on the side of caution here.”