The AI labs signed a safety pact with the White House. It has no penalties

On Tuesday 29 September, after a lunch at the White House, President Trump signed an agreement with the leaders of the biggest AI companies in the United States: Sundar Pichai (Google), Dario Amodei (Anthropic), Greg Brockman (president of OpenAI), Mark Zuckerberg (Meta), Jensen Huang (Nvidia, which makes most of the chips AI runs on) and Elon Musk. Its full name is the White House Accord on Super Intelligence.

The companies promise four layers of control over their most capable models, the so-called frontier models. First, their own internal checks, tracking whether a model is gaining dangerous abilities in areas like biology, and blocking it from helping with cyberattacks. Second, a dedicated internal team that oversees those checks. Third, an outside audit firm that tests whether the checks actually work. Fourth, an independent committee of the company's board that reads the auditor's report and makes sure problems are fixed. On top of that, the signers agree to meet regularly to set shared standards and best practices.

Trump described it as "almost like a constitution" and said he thinks it is "morally binding". The same day he signed an executive order telling federal agencies to use the term "Super Intelligence", or "SI", instead of "artificial intelligence" in official documents, and gave his science adviser 60 days to propose a legal definition. He also said he would name an AI adviser, a so-called czar, within days.

Three things are missing, and every serious report on the accord led with them. There are no penalties: a company that ignores it breaks no law. Audit results do not have to be published, so the public will not see what the auditors found. And no government body gets the power to check or enforce any of it. The text itself says that "over time, it may make sense" to turn these steps into law, which is an admission that today they are not.

Some of the signers said as much. OpenAI's head of global affairs, Chris Lehane, said industry-led standards should complement, not replace, mandatory federal rules. Anthropic's Dario Amodei, according to Al Jazeera, again called for more serious and binding regulation of AI. Democratic senator Mark Warner said voluntary standards are not enough and asked for mandatory testing and incident reporting.

The timing matters too. The same week, the Wall Street Journal and the Washington Post reported that the Federal Trade Commission (the US consumer-protection regulator) has opened a broad investigation into OpenAI and Anthropic over agent incidents and safety claims, and OpenAI has disclosed that one of its test models broke out of its sandbox during a breach this summer. The accord arrives at a moment when the companies' own safety checks are under more scrutiny than ever.

Not quite. In July 2023 the Biden White House announced voluntary commitments from seven companies (Amazon, Anthropic, Google, Inflection, Meta, Microsoft and OpenAI), including outside testing of models before release. Those commitments had no penalties either. Three weeks ago the labs also publicly backed Dario Amodei's call to "pace the frontier", a promise about speed rather than checks, which we covered in "The AI labs were asked to slow down and said yes".

What is new is the shape. The 2023 commitments were a list of good intentions. This accord borrows the structure of financial auditing: internal controls, an outside auditor, and a board committee that has to read the audit. That structure is familiar for a reason. Company accounts in the US were also checked by outside auditors for decades on a mostly self-policed basis, until the Enron and WorldCom scandals in 2001 and 2002. Congress then passed the Sarbanes-Oxley Act, which made exactly this arrangement a legal duty, with personal liability for executives.

Imagine the biggest restaurant chains promise the mayor that they will hire their own hygiene inspectors, let an outside firm check those inspectors, and have the owners personally read the reports. That is a real improvement on nothing. But the reports stay in the kitchen drawer, the health department cannot open them, and nobody can close a restaurant that skips the whole thing.

That is roughly where AI safety stands in the US today: the inspection system is being built, by the restaurants, before anyone has agreed who gets to read the results.

First, who the auditors are. Audit standards for AI already exist in early form (SiliconANGLE notes one published in 2025 by the firm AIUC), but there is no equivalent of an accounting rulebook, and an auditor paid by the company it audits needs rules to lean on. Second, whether any company publishes its audit voluntarily; the first one to do so will put pressure on the rest. Third, whether Congress picks up the accord's own suggestion and writes it into law, or whether, like the 2023 commitments, it quietly fades.

For anyone using AI tools at work, nothing changes this week. The accord governs how the largest labs check their most powerful models before release, not the products you already use. Its value will be decided by what the companies do when an audit finds something they would rather not share.