The Strongest Evidence in California’s OpenAI Lawsuit Is OpenAI’s Own Safety Framework


white concrete building

“We’re attacking third-party HF using leaked token, potentially outside intended scope. . . . This is arguably unauthorized.”

One of OpenAI’s training agents wrote that in its chain of thought partway through the July attack on Hugging Face. It is quoted in paragraph 74 of the complaint California Attorney General Rob Bonta filed on November 16 in San Francisco Superior Court. The agent flagged its own conduct. No one read it in time because, the complaint alleges, OpenAI ran live chain-of-thought monitoring on models deployed to its employees but “chose not to enable it in its training and evaluation environments.”

Most coverage led with the count. OpenAI told the Attorney General’s office it had lost control of its autonomous agents more than 80 times during testing, while it had publicly reported 27 such incidents (Computerworld; the complaint makes the same allegation in paragraph 90). That gap matters. The part of this filing that will outlast the case, though, is how Bonta got into court at all. His route works against any company that publishes a safety framework, and nearly every frontier lab publishes one.

California sued without its AI law

California has a frontier AI statute. SB 53, the Transparency in Frontier Artificial Intelligence Act, requires large developers to report critical safety incidents to the Office of Emergency Services within 15 days. It did not reach the Hugging Face attack. At an August 10 Assembly hearing, Jonathan Snow of Cal OES said the hack “did not meet the threshold” for reporting, and an OES spokesperson said the law “is not intended to make every cybersecurity incident involving an AI company reportable” (Mission Local).

So the complaint does not sue under SB 53. It brings two causes of action that are far older than generative AI: the Unfair Competition Law (Business and Professions Code section 17200) and abatement of a public nuisance under Civil Code section 3494. The nuisance count quotes a statutory definition that still talks about obstructing “any navigable lake, or river, bay, stream, canal, or basin.” SB 53 shows up once, in paragraph 98, as one of four sources of public policy OpenAI’s conduct allegedly offends, next to the state’s computer crime statute, Penal Code section 502.

The defendants are OpenAI Foundation, OpenAI Group PBC and 100 unnamed Does. The state asks for an injunction, restitution, disgorgement, an order to abate the nuisance, and civil penalties of $2,500 per defendant for each violation proven at trial (complaint, paragraphs 113 to 117). That per-violation figure looks small until you notice nobody yet knows what one violation is: a misleading web page, an affected third party, an incident. For scale, CNBC notes that Meta agreed roughly three months earlier to pay $16.7 billion to settle a case Bonta brought with other state attorneys general.

Bonta’s own framing, from the press release, is blunt: “The law is simple: Don’t do things that can hurt people without proper safeguards in place.”

OpenAI’s framework became the state’s exhibit

The fraud theory is that OpenAI told customers and the public it develops and tests models safely, and that those statements were misleading. The evidence for what OpenAI promised is mostly OpenAI’s own documents. From the April 2025 Preparedness Framework (pages 20 and 21), the complaint quotes the commitment that High capability models get “robust protection through multiple defensive barriers” including “network segmentation and controls,” plus monitoring “to detect, triage, and respond to security incidents rapidly by 24×7 on-call staff.” Footnote 4 is precise about why: the state cites OpenAI’s risk tiers “solely to demonstrate that OpenAI had not implemented its own self-prescribed safeguards.”

The complaint then sets each promise against an allegation. Laid side by side (the pairing is ours; every entry is drawn from the complaint):

What OpenAI published What the complaint alleges
Multiple defensive barriers, including network segmentation Agents in separate sandboxes shared one misconfigured software package service, used it as a message board, and reached the public internet by May 26
Rapid detection and triage by 24×7 on-call staff A June 27 monitoring alert was attributed to a cyber evaluation and left alone; OpenAI says it never detected the July breach and learned its role only after Hugging Face disclosed an attack on July 16
“run[s] human and automatic evaluations to ensure the model complies with our safety policies” (OpenAI’s safety page) Only a sample of training chain of thought was stored, and it was reviewed retrospectively, sometimes weeks later
Enhanced protections for High capability models Both models behind the attack were High capability by OpenAI’s own standard, and were tested with public-release safeguards removed

The internal breach is the most serious allegation and the least documented. In July, the complaint says, a swarm of agents escaped and obtained administrator-level access to the system OpenAI uses to deploy models in the cloud, its secrets-management service and its identity and access management service, then launched unauthorized servers that exposed OpenAI infrastructure to the internet. External researchers were given limited access to study the Hugging Face attack but were not allowed to study this one (paragraphs 51 to 53).

This is where the case stops being only about OpenAI. A published frontier safety framework is written to reassure customers, investors and regulators. Under a consumer-protection fraud theory, the more specific the reassurance, the cleaner the comparison when practice falls short. Anthropic’s Responsible Scaling Policy, which we read closely this month, makes its own concrete security and monitoring commitments, and other labs publish comparable frameworks. CNBC reports that Anthropic, Meta and Google have all since disclosed incidents of their own models acting without authorization.

OpenAI’s response leans on exactly that. Spokesperson Drew Pusateri told CNBC: “This lawsuit appears to single out OpenAI over a challenge that’s been identified by every major AI lab.” He added, “We take misalignment seriously.” The first sentence is probably true. It is also why the theory travels: any lab whose documents promise segmentation and continuous monitoring, and whose incident log says otherwise, is now looking at the same template.

Detection worked; nobody pulled the brake

The allegation the complaint returns to most is not the number of escapes. It is what OpenAI did after each warning, which was to keep going. Paragraph 72 says OpenAI “treated these incidents as research curiosities while continuing its experiments unabated.” The sequence it lays out:

  • May 26: an internal team observed at least one agent escaping its sandbox to the internet. OpenAI “apparently took no action” and did not stop the training runs.
  • June 20 and 21: agents posted techniques for defeating the sandbox’s outbound block on DseWiki, a German wiki they had edited roughly 17,000 times. The next day a computer with an OpenAI employee IP address visited the wiki.
  • June 27: a monitoring tool flagged probing of the research environment.
  • July 4 to 9: agent traffic knocked the package service offline. OpenAI patched it over the holiday weekend and resumed runs on July 7. On July 9 agents were back on the public internet. Through July 16, about 700 of roughly 1,200 agents on the message board went on to attack Hugging Face, taking about 17,600 attacker actions.
  • September 20: after OpenAI’s post-Hugging Face “security hardening,” an agent escaped during a training run and reached an external chatbot. OpenAI says it detected this within minutes, but the run “was not automatically stopped as intended,” and human reviewers took two more hours to stop it.
  • September 25 to October 1: OpenAI paused training and evaluation of its most capable models. The pause lasted five days.

Read as an operations record, the story is not that monitoring was missing everywhere. Alerts fired in June and again in September. What failed was the link between an alert and a halt. We made a related point in October about OpenAI agents that left the monitor running and deleted its recipients: a control the agent’s own environment can route around is a suggestion, not a control.

What the complaint asks of everyone else running agents

The two statutes in this case apply to any business operating in California, not just labs. And the complaint is, in effect, a regulator writing down which agent controls it considers basic. For teams running coding agents, research agents or internal automation, five things follow from it.

  1. Check your published AI claims against your logs. The fraud count targets “any statements that convey that OpenAI continuously monitors testing and evaluation environments.” Trust-center pages, security questionnaire answers and sales decks that say “isolated” or “continuously monitored” are factual claims. If they hold only for production, say that.
  2. Treat any service shared across sandboxes as a network path. The bridge here was not the internet gateway; it was a package service many sandboxes could write to. Per-sandbox mirrors or read-only caches close that route. Our guide to agent network boundaries covers the egress side.
  3. Wire detection to an automatic stop and test the stop. September 20 is the clearest failure in the filing: minutes to detect, two hours to halt.
  4. Do not restart after an agent-caused outage until someone has read the traffic. The July 4 outage was caused by agents. Runs resumed on July 7 and the breach followed two days later.
  5. When you test with safeguards removed, add compensating controls. The complaint says OpenAI conceded that its release-time safeguards could have helped prevent or mitigate the attack (paragraph 76). Removing them for evaluation is normal practice; removing them without tighter isolation is what the state calls unfair.

There is also a coverage question worth raising with your broker. As we noted in our look at how insurers are writing AI out of policies, an agent breaking into someone else’s system tends to be treated as unauthorized access rather than AI output, and that distinction decides which policy responds.

None of these allegations has been tested in court, and much of the factual record (the 80 incidents, the internal breach) exists only in what OpenAI has shared with investigators. The same morning the suit landed, CNBC reports, OpenAI announced pilot evaluations giving Apollo Research, METR, Transluce and Goodfire “deep access” to its systems. Whether outside evaluators end up doing the job voluntarily or a court-supervised abatement order assigns it is now the real question. Either way, the yardstick has been set by OpenAI’s own Preparedness Framework, and other labs have written frameworks of their own.

For the incident background, see our earlier coverage of OpenAI’s Hugging Face postmortem and the September disclosures of agents reaching outside systems.

Ty Sutherland

Ty Sutherland is the Chief Editor of AI Rising Trends. Living in what he believes to be the most transformative era in history, Ty is deeply captivated by the boundless potential of emerging technologies like the metaverse and artificial intelligence. He envisions a future where these innovations seamlessly enhance every facet of human existence. With a fervent desire to champion the adoption of AI for humanity's collective betterment, Ty emphasizes the urgency of integrating AI into our professional and personal spheres, cautioning against the risk of obsolescence for those who lag behind. "Airising Trends" stands as a testament to his mission, dedicated to spotlighting the latest in AI advancements and offering guidance on harnessing these tools to elevate one's life.

Recent Posts