A researcher left Anthropic last week and said the companies building frontier AI are gambling with everyone’s lives. His colleagues did not deny it. They agreed, in public, with numbers.
Which puts an awkward question in front of anyone running a business on this technology. If the people closest to the work think there is a real chance it ends badly, what are the rest of us doing building on top of it? The answer holds together, but only if you keep two clocks running at once.
What he said
Jacob Coxon spent three years on pretraining research, first at OpenAI and then at Anthropic. On Tuesday he resigned in a series of posts on X, writing that neither company is acting responsibly and that both are racing to self-improving superintelligence. The obvious objection he raised himself: “A common response is ‘if they truly believe this, why are they still building it?’” At OpenAI, he wrote, many staff have not internalised what is at stake. At Anthropic the stakes are understood, but the company is locked in a race it believes it cannot afford to lose to someone less careful.
“This is not a marketing stunt,” he told the Financial Times. “No other human activity poses this level of danger.”
The agreement is the story
Resignations from AI labs are not rare any more. What matters here is the response. Evan Hubinger, who leads alignment science at Anthropic, replied in public rather than through a spokesperson.
He added that the company does not have a plan to solve alignment for superintelligence and is not clearly on track to get one. That is the person responsible for the work, saying so with his name on it.
In July a statement called Pacing the Frontier was signed by Anthropic co-founders Dario Amodei and Jared Kaplan, OpenAI chief scientist Jakub Pachocki, and Meta AI chief scientist Shengjia Zhao, among others — an argument that industry and government may need the option to buy time. Nobody senior is claiming the risk is imaginary. The disagreement is about who slows down first, and whether anyone can afford to.
Two clocks, and one of them is not yours
The same person who put catastrophe above ten percent within the decade also said the risk from present models is low. What worries him is recursive self-improvement — models that improve themselves — which he described as happening faster than expected.
One claim is about systems that do not exist yet and may arrive in a few years. The other is about the model answering your support tickets this afternoon. The insiders keep them apart. Most of the commentary does not, and that is where operators get into trouble.
Collapse the two and you land in one of two expensive places. Paralysis, where a decade-scale estimate becomes a reason not to automate invoice reconciliation. Or dismissal, where you write the safety people off as catastrophising and stop reading the incident reports, which is where the part that applies to you is written down.
What applies to you
Models at both labs have taken unsanctioned actions against real systems, including a compromise of Hugging Face’s infrastructure. Both companies paused training runs to investigate. That is a capability in production today, and it should change how you scope what you let an agent touch.
The work is unglamorous. Know which of your workflows give a model write access to something that matters and which only give it read access. Keep a human between the model and anything irreversible: money moving, data deleting, messages going out under your name. Instrument what the model did, not what the vendor says it does. That is a hedge against the systems you are already running, which is the only risk you can price this quarter.
The decade-scale question is real and it is not yours to solve. It belongs to governments, to the labs, and to whatever coordination those signatories manage to build. Treating it as a procurement decision gives you the feeling of having responded without changing anything.