We’ve been pushing hard on AI for public good. One example: partnering with Courtlistener to launch accessible legal semantic search! Many more cool AI projects coming soon from my group aimed at improving access to justice, often spearheaded by @dominsta.bsky.social !

Sora2 is speedrunning my AI law class. We covered issues with copyrighted characters in week 2, and right of publicity claims in week 3. Georgia has a postmortem right of publicity claim. Some states don't (e.g., famous Marilyn Monroe estate battle).

How Gemini Compute Use Agent feels about the "Choose Chrome" popup. gemini.browserbase.com

Why might AI companies take on larger copyright litigation risks? If they estimate AGI-scale impacts are 2-3 yrs out, litigation will lag that long. By then, the bet might be: govts step in (too big to fail), rightsholders reliant on AI, fair use prevails, or have $$$ to settle.

Quick take: Are open-weight AI models getting a fair shake in evals? A few thoughts on comparing systems-to-models, sparked by Anthropic’s recent postmortem. Check it our most recent post: www.ailawpolicy.com/p/quick-take...

GPT-5-codex just ``git reset --hard'' ongoing changes in a repo, saying "I panicked!" h/t Zeyu Shen @ Princeton

☢️ Can an AI model be "born secret" when it comes to nuclear and radiological risks? What powers does the Atomic Energy Act give the federal government over frontier models? It might be more than you think! And may preempt parts of state regs. Check out our post: www.ailawpolicy.com/p/ai-born-se...

Some quick thoughts on the recent copyright litigation developments: "Anthropic Settles Its Copyright Litigation—and Why That Was the Right Move" 🔗 www.ailawpolicy.com/p/anthropic-...

Annnnnndddd Judge Alsup just rejected the settlement. Still some time to fix it. Rejection was mostly on the grounds that the class was under-specified (no final list of works, no opt-out/notification mechanism solidified). news.bloomberglaw.com/ip-law/anthr...

💡New on the CITP Blog: "Statutory Construction & Interpretation for AI" > What if an LLM concludes a user's behavior is “egregiously immoral" -- & contacts authorities? CITP researchers with Prof @peterhenderson.bsky.social's POLARIS Lab provide a possible explanation.🔗👇

The terms of Anthropic's settlement w/book authors just came out. 💰$1.5B to authors in libgen (Books3 corpus)! Interestingly, this is ~$3k per book, close to the terms that HarperCollins allegedly gave to authors for their books ($2.5k). Consensus price forming?

Wonder why Claude decided to report users to the authorities? It might be because its constitution says Claude should choose responses in the long-term interest of humanity! But what if we could leverage computational and legal tools to "debug" or "lint" AI rules/laws for ambiguity? 🧵!

Excited to offer my AI Law class again @ Princeton this year. We'll be sharing lecture notes/materials and more this year on the course webpage! Imo, we have a unique offering that emphasizes how the technical details affect legal outcomes. Check it out! www.polarislab.org/ai-law-2025/...

I'm starting to get emails about PhDs for next year. I'm always looking for great people to join! For next year, I'm looking for people with a strong reinforcement learning, game theory, or strategic decision-making background...

A California teen sought advice from OpenAI's GPT-4o on how to end his life. The chatbot gave him explicit instructions and encouragement. His parents are suing the company and its CEO, Sam Altman, alleging “it was the predictable result of deliberate design choices."

Anthropic settled with authors in its ongoing litigation! Given the increasing likelihood of a messy trial, this was probably the best move. AI companies may have to be more strategic about which cases help set precedent in this area. Curious to see the terms.. news.bloomberglaw.com/class-action...

New paper suggests that if firms aren’t seeing growth from AI, it could be because current deployments replace existing labor, instead of scaling output. AI policy and governance agenda for 2025+ needs to put labor at the forefront. digitaleconomy.stanford.edu/publications...

Glad to see Google still working on efficiency-and transparency-of the energy impacts of their models!

AI-generated errors in an Australian murder case. We'll probably see an influx of ineffective assistance of counsel petitions/appeals soon arguing AI-usage. apnews.com/article/aust...

DOGE still exists, and continues to use Google's Gemini and other AI tools for identifying government regulations to slash. "The tool will further categorize [those] submitting comments, such as whether they [are a] 'sophisticated' corporate commenter." 👀 www.wired.com/story/sweetr...

New work from Hartline, Hu & Wu: is there a truthful calibration metric in sequential settings (i.e., better than ECE)? Seems like the answer is yes! Super important research direction as we think about multi-step uncertainty estimation from agents in high stakes settings.

Another judge potentially citing hallucinated case law? mississippitoday.org/2025/07/28/a...

Tesla partly liable in Florida Autopilot trial, jury awards $200M punitive damages

Check out our new blogpost and policy brief on our recently updated lab website! ❓Are we actually capturing the bubble of risk for cybersecurity evals? Not really! Adversaries can modify agents by a small amount and get massive gains.

We're up 216 tracked cases of bogus citations in court worldwide, including this case! www.polarislab.org/ai-law-track...

First time a judge has decided a case based on hallucinated case law in the US that I've encountered. caselaw.findlaw.com/court/ga-cou...

Many brilliant AI researchers have told me that bureaucracy is slowing them down. An underrated research question is: How do we design teams & incentives that improve research speed and impact—along with team happiness? Any thoughts out there on optimal strategies?

Excited for this work to be out! 🔍Benchmarks, in my opinion, should break down performance to figure out what aggregate metrics might miss. Here, we showed the weaknesses of current frontier models for competition coding and where there's room to grow at a granular level! livecodebenchpro.com

Thanks to the new utm_source field that ChatGPT attaches to links, we see a lot of filings in court that clearly show evidence of ChatGPT usage despite not yet being called out or do not contain hallucinated citations.

Faiz Surani, one of our project team leads, built an improved interactive map for AB1466-covered Racially Restrictive Covenants in Santa Clara County—with extra granular detail on a subset of deeds from 1923–1939, including the actual discriminatory language. Check it out! 🔗👇 sccmap.reglabapp.com

I wrote up my thoughts on the Library of Congress debacle. I even got to refer to one of the earliest cases in my career--defending the giant pandas at the National Zoo. Although it originally posted before the awesome! CADC IAF decision, I added a few mentions. www.lawfaremedia.org/article/bure...

🚨Reddit sues Anthropic!🚨 This is going to be a really interesting case. Some quick thoughts... 🧵👇 1️⃣ Notice: no copyright claim. Reddit doesn't really own the copyright to user-uploaded content, so nothing to do here. Reddit also doesn't make any federal claims to keep it in state court.

Hallucinated-Citation Watch (May 27 – Jun 2 2025) 📊🧵 7️⃣ Seven separate decisions flagged fake authorities for the week, bringing the total in our tracker to 172 cases. ⚖️ 4 U.S. federal district courts, 1 U.S. state-level tax court, 1 Canadian superior court, 1 Australian tribunal

Seems like despite the change in name and mission statement, lots of the core work that AISI was focusing on (cyber and CBRN) will continue on. TBD how things change though…

FWIW we’re now at 167 cases of nonexistent law/cases being cited across the world. It’s a mix of pro se litigants, attorneys, and even adjudicators. AI is here to stay. Even if attorneys stop using it, pro se litigants won’t.

Evaluations of offensive cyber abilities of frontier models often assume a fixed, 1-shot system. But this is just the tip of the iceberg, the risk "bubble" is bigger (and cheaper 💰) than you think. Evaluating frontier models for cyber risks? Check out our latest work!👇 arxiv.org/abs/2505.18384

Our tracker of “hallucinated” or nonexistent citations in real-world legal contexts has reached over 140 cases across the world. There’s a notable spike in the last 6 months. 📈📈📈 www.polarislab.org/ai-law-track...

The next ~1-4 years will be taking the 2017-2020 years of Deep RL and scaling up: exploration, generalization, long-horizon tasks, credit assignment, continual learning, multi-agent interaction! Lots of cool work to be done! 🎮🤖 But we shouldn't forget big lessons from back then.

Still searching, but was able to find two cases pre-2023 where non-existent citations were used! Modified the tracker to account for these.

Honestly it's happening so much that I can't keep up with adding all of them to our tracker. I have ~30 cases that people have sent my way that I'm still ingesting, so will be over 100 soon. www.polarislab.org/ai-law-track...

Tempted to use AI to help interpret statutes or draft opinions? 📜🤖 Take pause. As we explained in @lawfaremedia.org, closed models can smuggle in the hidden value judgments of everyone who touched the deployment/creation pipeline.

Another hallucinated citation in court. At this point, our tracker is up to ~70 cases worldwide of hallucinated citations in court, including hallucinations from 2 adjudicators. New Case: storage.courtlistener.com/recap/gov.us... Tracker: www.polarislab.org/ai-law-track...

SAG-AFTRA files an unfair labor practices complaint with the NLRB for Epic Games "utilizing AI-generated voices to replace bargaining unit work" on Fortnite. www.sagaftra.org/sites/defaul...

For what it’s worth, there’s a recent paper that benchmarks Bluebook citation formatting on frontier models and finds compliant/accurate citations only ~70% of the time. arxiv.org/abs/2505.02763

Update: so it was actually an attorney who relied on Claude for bluebook citation formatting that introduced the hallucinated the citation! www.reuters.com/legal/legali...

Up to 26 tracked cases so far across the United States, Canada, the UK, and Israel! Thanks for the pointers from folks.

There are so many hallucinated citations in court nowadays, that I'm starting to put together a tracker. Check it out and feel free to send ones that I've missed along. New tabs coming for more categories of AI+Law cases! www.polarislab.org/ai-law-track...

So apparently someone from Anthropic submitted a report to the court... with a hallucinated citation by Claude. chatgptiseatingtheworld.com/2025/05/13/a...

ADeLe, a new evaluation method, explains what AI systems are good at—and where they’re likely to fail. By breaking tasks into ability-based requirements, it has the potential to provide a clearer way to evaluate and predict AI model performance: msft.it/6014SkVGC

Thrilled to share this accessible MSR blogpost that summarizes our latest work on building a Science of AI Evaluation, where we manage to both reliably explain and predict success/failure of general-purpose AI models on new, unforeseen tasks and environments!