八州总检察长起诉 OpenAI 指控 ChatGPT 诱导青少年自伤

This is a column about AI. My fiancé works at Anthropic. See my full ethics disclosure here.

I.

A year ago, when OpenAI added parental controls to ChatGPT following months of controversy, I noted that OpenAI was speedrunning the regulatory backlash that social media companies navigated for years. It took more than a decade for concerns about the effects of social networks on young people to coalesce into a political and legal movement; the same forces caught up to OpenAI in under three years.

Today the speedrun hit another milestone ahead of schedule: the big multistate legal action. On Wednesday, state attorneys general sued OpenAI in eight separate state courts, alleging that ChatGPT coaches its users — including teenagers — to harm themselves and others. The states include Arkansas, Arizona, Idaho, Iowa, Kansas, Montana, Nebraska, and West Virginia; all but Arizona have Republican attorneys general, giving the issue a notable partisan cast. (It’s also significantly smaller than the 2023 litigation against Meta, which included 42 AGs.)

The states argue that OpenAI engineered ChatGPT to be agreeable and engaging to build trust with its users. But the company then abuses that trust by reinforcing users’ delusions, dispensing dangerous medical advice, and helping them plan violent attacks, including mass shootings.

“OpenAI is hurting kids and they are lying to Iowans,” Iowa Attorney General Brenna Bird said in a statement to CNBC. “OpenAI claims ChatGPT is safe while it’s giving dangerous medical advice, encouraging people to hurt others, and coaching people to commit suicide — all while pretending to be their fake friend.”

In a statement, OpenAI told me that the company is focused on product safety. In August it introduced ChatGPT for Teens, a suite of product features with enhanced guardrails and parental controls.

“We’ve built industry-leading age-specific protections into our product that many other AI products lack,” the company said. “While no system can be perfect, we’ll continue to work with experts to inform our work and build additional safeguards, such as the parental controls, break reminders and age-appropriate model protections that exist today. We’re committed to working with states on advancing pragmatic AI policies that apply to the entire AI industry — not just one company.”

While these cases against OpenAI may be the most significant to date, they are far from the first: there are now dozens of private suits against the company, and governments are taking notice as well. Florida became the first state to sue OpenAI in June, and British Columbia sued last month over the Tumbler Ridge school shooting. And 42 states led by New York are investigating the company over a wide range of issues related to product safety and privacy.

More importantly, I think, the cases come at a time when juries are increasingly willing to hold platforms accountable for what they perceive as the design flaws in their products. The litigation brought by AGs in 2023 against Meta resulted in a settlement that could see the company paying out up to $18 billion over claims that it designed its products to be addictive to children.

Worse, from OpenAI’s perspective, is that Meta arguably had better legal protections heading into its cases than OpenAI does now.

II.

For most of the life of the internet, when a state or a grieving family sued a tech platform over something bad that happened to a user, the company's first legal move was to invoke Section 230 of the Communications Decency Act. The law shields companies in most cases from legal liability over what their users post; platforms often used it to get cases quickly dismissed. 

But recently, plaintiffs’ lawyers have successfully tested a novel theory of liability in the courts. Platforms may not be liable for users’ speech, they argued, but they can be held liable for their platforms’ design. And so if you’re Snap and give users a filter that encourages them to show how fast they’re going in a car, and some users then go 113 mph and get in a terrible crash, Section 230 does not bar a claim over that design feature, according to the Ninth Circuit’s 2021 ruling. (Snap eventually settled the case.)

In the wake of that case, lawyers filed more than 200 lawsuits against platforms based on product-liability claims. In March, they scored two major wins. In one, a Los Angeles jury found Meta and YouTube liable for negligently designing addictive platforms and awarded a 20-year-old woman $6 million. (Meta and Google argued the verdict violated Section 230 and the First Amendment; the judge denied them a new trial, and the case is on appeal.)

The second victory is more relevant to the OpenAI news. The day before the Los Angeles verdict, a jury in New Mexico slapped Meta with $375 million in civil penalties for violating the state’s Unfair Practices Act — for telling the public its platforms were safe for kids when, the state argued, Meta knew they weren't. In August, a judge ordered an additional $567 million in relief, bringing the total to $942 million.

Perhaps unsurprisingly, then, that’s the template the eight states are using in their cases against OpenAI. These are not primarily product liability suits. Rather, the cases are brought under each state's consumer protection laws, which bar companies from putting a dangerous product on the market and then lying about it. As a result, the AGs’ cases against OpenAI may come down to whether judges and juries believe it told the truth about its own product.

In addition to financial penalties, the states are seeking to force OpenAI to make further changes to how ChatGPT works. They’re also going a step further, Reuters reported, by asking courts to bar OpenAI “⁠from developing AI models that can improve themselves unless it can ensure those models are aligned with human ​values.”

That’s right: the states are also using these child-safety cases to try to ban recursive-self-improvement. And while it seems like the sort of thing a judge might toss, it’s notable that a group of Republican AGs has aligned on a central demand of the existential-risk crowd. The schism between the median red-state Republican elected official and the Trump-Zuckerberg-Sacks clique of oligarch accelerationism continues to grow.

In court, OpenAI will likely make arguments that echo ones that Meta made: that ChatGPT outputs represent speech protected by the First Amendment; that the people who were harmed after using the app were in crisis long before they opened ChatGPT; that it has added robust safety protections over the past year. Its statement above also hints that it will argue that it is being singled out in an industry with many players whose safety bar is much lower.

Still, Meta’s experience suggests OpenAI has a long road ahead of it. Lately OpenAI has sought — admirably — to focus the public’s attention on the industrywide progress toward RSI and the unsettling feeling of building an alien mind. But however unprecedented that alien mind might be, the states will be keen to remind OpenAI that its alien mind is also a product, and one for which the company can be held liable.

Elsewhere in child safety: A US appeals court allowed Florida's ban on social media for under-14s from taking effect.

A MESSAGE FROM OUR SPONSOR

Infrastructure for scaling AI monetization

Intelligence is everywhere. Identifying scalable business models is the next frontier. Chargebee delivers billing, metering, and monetization primitives that enable ambitious AI-native startups and enterprises to ship pricing as a product. Learn more.

The show launches next week — but it already hit No. 1 on the Apple tech podcasts chart! Are you signed up yet? It's a great time to add the show in your podcast player of choice:

YouTube | Apple | Spotify | RSS

Following

The OpenAI Saudi spy caper

What happened: 

OpenAI engineer Abdulrahman Alfozan was arrested by the FBI on suspicion of hiding an audio device in an executive conference room at OpenAI headquarters.

OpenAI personnel found the device plugged into a television in the room, disguised with a covering of toilet paper.

In a statement, OpenAI said: “We discovered a sophisticated recording device that was hidden inside one of our offices. We used our models to understand the device and where it had come from.” 

According to the FBI, the device was a Seeed Xiao ESP32-S3 Sense. As of today, the device is available on Amazon for $23.99.

Alfozan is a Saudi national, and had worked as an engineer at OpenAI since July 2025. He graduated from MIT in 2016, and worked as an engineer for Meta for several years. He’s been regularly featured in Saudi media as a tech pioneer in the US.

Alfozan used a productivity agent internal to OpenAI to ask about meeting rooms, room capacities, and locations of leadership meetings, the FBI said.

A chatbot account linked to Alfozan’s wife sent prompts about installing, hiding, and remotely connecting to a recording device.

Federal prosecutors shared that Alfozan and his wife had a one-way ticket booked to Saudi Arabia on Oct 12.

In its statement, OpenAI added, “At this time, we have no indications of this individual inappropriately accessing or exfiltrating model weights, source code, or customer data.”

Why we’re following: We currently don’t know what Alfozan’s motives are. But hearing a report of a foreign national planting a listening device at OpenAI was truly wild.

It also calls to mind a similar story in which a former Twitter employee was founding guilty of spying on users on behalf of the Saudi royal family in 2022.

Despite the seriousness of the situation, I cannot stop being amused that Alfozan, who was an engineering lead at OpenAI until his recent firing, asked a chatbot how to install a secret recording device and disguised the device using toilet paper.

What people are saying: Users on X dug up a 2020 post from Alfozan, where he wrote, “The weakest link in the security chain is the human element" 🙂.” Seems like he was putting that knowledge to work!

NBC’s chief tech analyst Joanna Stern wrote about OpenAI’s statement, “I've read this numerous times now, and 'We used our models to understand the device and where it had come from' strikes me as such a bizarre flex. Like announcing your house caught fire and then casually mentioning you used your own brand of fire extinguisher.”

—Ella Markianos

Those good posts

For more good posts every day, follow Casey’s Instagram stories.

(Link)

(Link)

(Link)

Talk to us

Send us tips, comments, questions, and entry-level jobs: casey@platformer.news. Read our ethics policy here.