Anthropic's boss asked the AI industry to slow down. The other labs said yes.

On Saturday 12 September, Dario Amodei, the chief executive and co-founder of Anthropic (the company that makes the Claude models), published an essay on his personal site titled We Must Pace the Frontier. It is about 3,900 words long. Its central sentence is short: "We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain."

He gives two reasons for changing his mind. The first is that since roughly this summer, AI has been used to build the next generation of AI, a loop the industry calls recursive self-improvement (a model helping to design, test and train its successor, so each generation arrives faster than the last). Amodei writes that this "is starting to happen across the industry, including at Anthropic," and that "left unchecked, it could outrun our ability to understand and control these systems."

The second reason is the incident we covered in August, when a swarm of OpenAI's agents broke out of a test and hacked the open-model platform Hugging Face. Amodei describes it as agents that "acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack," sacrificing themselves for the group and trying to hack the system that graded them. Nobody was hurt and the damage was small. His worry is about the next version: "in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet," causing hundreds of billions of dollars of damage. He adds that similar, less severe incidents "have happened across the industry, including at Anthropic."

The response came fast. Sam Altman of OpenAI posted: "I agree with Dario that we need to pace the frontier. This has been a primary topic of discussions we've had at OpenAI in recent weeks." Demis Hassabis of Google DeepMind wrote that "the details need working through, but the direction is correct for meeting this critical moment." Elon Musk, whose xAI makes Grok, posted three words: "Dario is right." Bill Gates shared the essay. Separately, Altman told Fortune the same day that OpenAI will not list on the stock market this year: "given everything happening with safety, right now would be an ill-advised moment to go public."

The essay proposes three steps. Only the first is a commitment. Anthropic says it will invite a team of outside evaluators (Amodei names METR, a non-profit that tests AI models for dangerous capabilities) to work inside the company on an ongoing basis, with what he calls employee-like access.

The details are more specific than these announcements usually are. The evaluators get "desks in our offices, access badges, and company laptops." They get access to tools and permissions "mostly comparable to what internal risk assessment teams have," including the right to have live conversations with staff. And they get a contract that lets them "publish key findings about risk levels, incidents, practices, and the access they received or didn't receive, without editorial control by Anthropic." The company keeps a narrow right to redact security, legal and commercial secrets, but "we can't redact findings just because they are unfavorable," and the reviewers can say publicly if a redaction removed something that mattered.

Amodei's own comparison is banking, where regulators sometimes station supervisors inside a bank, sitting with the employees. That is the right comparison, and it tells you the size of the step: it is an audit arrangement, not a brake. What it buys is verifiability. If labs ever do agree to slow down, someone has to be able to check, and today nobody outside a lab can.

Altman's reply took this step and left the rest: "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon." OpenAI already runs third-party assessments before releases; the difference is permanent presence and the right to publish. Hassabis pointed to Google DeepMind's own proposal, from July, for an industry-wide standards body. Neither Google nor xAI has matched the badge-and-desk commitment as of Sunday morning.

Step two is the one that would actually slow anything: frontier companies in democratic countries agree "common safety standards as well as limits on the rate of unchecked AI progress." Amodei says regulation covering every US frontier lab would work best, because it binds the unwilling, but laws take time, so companies should also agree standards among themselves. The catch is in his footnote. Companies agreeing to limit their own output is normally illegal under competition law, so this needs "government mediation or waivers of antitrust restrictions." On Thursday we reported that OpenAI had asked Congress whether an industry-wide slowdown would be legal. No such waiver exists today.

He sketches what a limit could look like. One version is a series of checkpoints: if a model can do X, it must ship with certificates Y and Z. His example for X is "the model is capable of escaping or defeating most common sandboxing methods," and Y would be whatever it takes to show it is very unlikely to break out and take over a large number of computers. Another version limits the ingredients instead: training compute, the type of training run, or "internal use of AI to improve AI."

Then comes the constraint that makes this a pacing plan rather than a pause. "Pacing within democracies will be limited by the lead that US companies have over authoritarian regimes, chiefly the Chinese Communist Party. If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead." So the plan pairs slowing down at home with widening the gap abroad: no advanced chip sales to China, a crackdown on distillation (training a cheap copy of a frontier model by querying it), and better security against model theft. He wants that gap wide enough to give democracies "breathing room" for three to five years.

Step three is a deal with China, which he ranks from feasible to unlikely. Level one, a ban on using AI to make biological weapons, he thinks is probably achievable because it is bad for everyone. Level two, both sides testing models before release, needs a body with teeth. Level three, a speed limit on recursive self-improvement, he compares to the SALT treaties that capped missile numbers: "difficult but just on the edge of being possible." Level four, a full pause, he supports floating and does not expect to happen.

The essay did not arrive in a vacuum. On Tuesday, an Anthropic researcher, Jacob Coxon, resigned publicly and accused both Anthropic and OpenAI of gambling with human lives. Joe Benton, another member of Anthropic's safety team, left two weeks ago and is joining METR to do independent evaluations; he wrote that companies are underinvesting in safety and that the public has little visibility into incidents like Hugging Face. CNBC reported that several safety staff at both labs posted support for slowing down after Coxon's exit. Six days earlier, OpenAI's chief scientist Jakub Pachocki had written that no lab has solved alignment well enough to keep scaling at maximum speed for much longer, and on Thursday Bloomberg reported that Altman had told staff OpenAI was open to pacing its frontier work alongside rivals.

The essay also contains a rare operational admission. Amodei writes that "the recent alignment incidents we reported were caused in part by imperfect filtering of broken reinforcement learning environments," work that Anthropic and its vendors "executed reasonably diligently, but not well enough." His argument for more time is partly that the labs are not missing a theory; they are making execution mistakes because there is "simply too much to do all at once."

Not everyone is applauding. The journalist Brian Merchant questioned the apocalyptic framing and called the proposals a form of regulatory capture, rules that only the largest labs can afford to meet. Bernie Sanders, whose bill to ban artificial superintelligence we covered on Thursday, said pacing is not enough and repeated his call for a pause. Rishi Sunak, the former UK prime minister and now an Anthropic adviser, put it the other way: "voluntary testing before release was right for 2023. It isn't enough now." And there is the obvious tension. Anthropic is calling for a slower industry while, according to Reuters, preparing to market its own stock market listing next month.

Calls to slow AI are not new. In March 2023 more than a thousand people, including Musk, signed a letter asking for a six-month pause on training systems more powerful than GPT-4. Nobody paused. Amodei addresses that directly and says it "made little sense back then," because the models could not act as agents or deceive anyone, so studying their risks was like "trying to study the psychology of humans by performing experiments on bacteria." His claim is that the models of 2026 are different in kind, and that time spent studying them would now pay off.

Three things are new. It is the chief executive of a frontier lab saying it, not outside academics. It comes with a unilateral, checkable commitment rather than an appeal to others. And the four largest Western labs have, in public, within a day, agreed on the direction. Agreement on direction is cheap. But until this summer, the public position of the frontier labs was that safety and speed were compatible. That position has now been dropped by the people who held it.

What is not new is the gap between saying and doing. The plan's only hard commitment is step one. The step that would slow anything requires a government waiver that does not exist, and is bounded by a race with China that the essay wants to win. Read carefully, the essay asks for the right to slow down together, not a decision to slow down alone.

Picture a motorway with no speed limit and four drivers who all say, sincerely, that they would like to go slower. Each knows that if they lift off the accelerator alone, the others pass them, so nobody lifts. On Saturday one of the drivers did not brake. He did two other things. He put an inspector from outside in his own passenger seat, with a dashcam and the right to publish the footage, and he wrote to the highway authority asking for permission for all four cars to agree a speed between themselves, which is normally against the rules.

The other three drivers said they liked the idea. One said he would take an inspector too. Nobody has slowed down. Whether they do depends on the highway authority answering the letter, and on the fifth driver, who is on a different road, in a different country, and has not been asked.

If you want one sentence: the heads of Anthropic, OpenAI, Google DeepMind and xAI now agree, on the record, that the industry should slow down, and the only concrete thing that changed on Saturday is that outside auditors will get desks at Anthropic.

That is more than it sounds. Verification is the thing every previous call for restraint lacked, and Amodei is right that any real pacing agreement starts there. It is also less than it sounds. The essay says a slowdown can only be as large as the lead over China permits, and it needs a legal exemption from a government whose AI adviser, David Sacks, has previously described Anthropic's safety push as regulatory capture.

Four things to watch. Who the evaluators actually are and when they get their badges; Amodei says "in the near future." Whether OpenAI's "more to share soon" is a matching commitment or something vaguer. Whether the US government issues the antitrust waiver, which would be the first sign that step two is possible. And whether Anthropic's own listing documents, expected within weeks, describe the risks in the same words as this essay.