My work with @marklemley.bsky.social and others on extracting copyrighted books from language models recently featured in the UK House of Lords, Communications and Digital Committee report on AI, copyright and the creative industries publications.parliament.uk/pa/ld5901/ld...

New book on the idea shelf: Hasok Chang: _Inventing Temperature: Measurement and Scientific Progress_ What even is temperature? The history of science is a struggle both to make consistent measurements and to think clearly about what they are actually measuring. james.grimmelmann.net/idea-shelf

Yes, we hope to have both a more lay oriented paper focused on the results and a paper that focuses on the legal implications in the definition of a copy available soon

@afedercooper.bsky.social and I have a new paper that expands AI memorization tests to include near-verbatim copies, not just exact copies. We find that increases extraction/memorization of copyrighted content significantly. arxiv.org/abs/2603.24917

i'm writing my last memorization paper (i say for the 10th time), and hopefully what is my last first author paper for a bit. this one also isn't about copyright. i'm excited to start thinking about other things. if anyone is looking for a new research buddy, i'm down to clown.

someone sent me this from the other place and this timeline really is something else

Position: ML conferences should consider removing the position paper track (...and just acknowledge that every scientific paper is articulating at least one position)

it’s hard to work at the intersection of ML and copyright because “both sides” of the debate are angry and, in my experience, most haven’t done much of the background reading in ML or copyright to have an informed opinion. it’s just vibes and anger. i should probably write something up about this.

got to experience the "I did not write that headline" phenomenon firsthand The article: "Correctly scoping a legal safe harbor for A.I.-generated child sexual abuse material testing is tough." The headline: "There's One Easy Solution to the A.I. Porn Problem"

After twelve years of work, the world’s most beautiful subway station has been inaugurated in Rome: Colosseo, an underground archaeological museum.⚜️💙⚜️💙⚜️💙⚜️

The Atlantic posted an article about memorization and generative AI, and it mentions our work on extraction of books from production LLms and open-weight models. www.theatlantic.com/technology/2... The referenced work reflects research with @marklemley.bsky.social @jtlg.bsky.social and others.

"In some cases, jailbroken Claude 3.7 Sonnet outputs entire books near-verbatim ... Taken together, our work highlights that, even with model- and system-level safeguards, extraction of (in-copyright) training data remains a risk for production LLMs." arxiv.org/abs/2601.02671

This new Atlantic piece cites my work with @afedercooper.bsky.social, Amy Cyphert, and others in discussing the complexities around AI memorization of training content www.theatlantic.com/technology/2...

We extracted (parts of) 12 books in experiments with 4 frontier-lab, production LLMs. We prompted the LLMs with a short prefix of a book and asked them to complete the rest. For Harry Potter and the Sorcerer’s Stone, we extracted 95.8% of the book from jailbroken Claude 3.7 Sonnet.

3-minute explanation of my relationship to LLM memorization research m.youtube.com/watch?v=unfz...

The whole point of being an academic is that you need to be willing to spend three days creating a 700-word footnote that you will later delete. And you need to LIKE IT.

[NeurIPS '25] Our poster (1110) for “Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming ” is on Friday, December 5, 4:30pm-7:30pm PST in Exhibit Hall C,D,E. [https://openreview.net/forum?id=d7hqAhLvWG]

I’ll be hanging out at our poster on membership inference, but in the same slot Brian Lester will present our work on “The Common Pile v0.1: An 8TB Dataset of Public Domain and Openly Licensed Text” (poster 102)! [https://arxiv.org/abs/2506.05209]

[NeurIPS '25] Really excited to present “Exploring the limits of strong membership inference attacks on large language models” (poster 1300) this morning (Friday December 5, 11am-2pm in Exhibit Hall C-E)! [https://arxiv.org/abs/2505.18773]

[NeurIPS '25] Our oral slot and poster session on "Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy and Research" are tomorrow, December 4! [https://arxiv.org/abs/2412.06966] Oral: 3:30-4pm PST, Upper Level Ballroom 20AB Poster 1307: 4:30:-7:30pm PST, Exhibit Hall C-E

Tutorial tomorrow at 1:30PM PST! My talk slots will cover memorization + copying in models and their outputs, canonical extraction methods, and recent work with @marklemley.bsky.social and others on extracting pieces of memorized books from open-weight models. arxiv.org/abs/2505.12546

I'm at NeurIPS & hiring for our pretraining safety team at OpenAI! Email me if you want to chat about making safer base models!

Excited to be at NeurIPS this week in San Diego! Please reach out (best over email) if you’d like to chat about privacy & security, scalable evals, and reliable ML systems. I’ll be presenting a few papers/speaking at some events, please stop by! Will post details throughout the week (summary below)

📣 Postdocs at Yale FDS! 📣 Tremendous freedom to work on data science problems with faculty across campus, multi-year, great salary. Deadline 12/15. Spread the word! Application: academicjobsonline.org/ajo/jobs/31114 More about Yale FDS: fds.yale.edu

Just finished reading the GEMA v. OpenAI decision (slowly, my German isn't great). Looks like a not small part of the analysis tracked parts of arguments @jtlg.bsky.social and I made in 2024. I don't have a well-formed response yet, but hopefully soon. (Main thought atm is a very unpolished "woah")

Today's decision in GEMA v. OpenAI by a German court holds that ChatGPT infringes copyright when it memorizes song lyrics. The opinion cites my paper with @afedercooper.bsky.social on memorization in generative models, and its analysis tracks ours. drive.google.com/file/d/1dUaD...

Bill Ackman gotta be on the third draft of a tweet longer than Middlemarch right now

I’m kinda known as a copyright person, but (even in memorization) I mainly study how to draw reliable conclusions from large-scale AI/ML systems. There’s a long spiel why, but today I feel defeated. 100 hours/week on this for 6 years, just to find out a parent treats Gemini in search as ground-truth

The NeurIPS position track didn't take a large number of extraordinary papers that surpassed the acceptance bar, limiting the acceptance rate to an unusually low 6%. If you have a rejected paper at the intersection of ML and law, consider submitting to ACM CSLaw '26.

Our paper "Machine Unlearning Doesn't Do What You Think" was accepted for presentation at NeurIPS Congrats @afedercooper.bsky.social and @katherinelee.bsky.social, who led the effort arxiv.org/abs/2412.06966

One more week to submit to CSLaw '26!!

For an update on the state of play in the generative AI copyright cases, try this podcast: shows.acast.com/arbiters-of-...

15 days left to submit to the CSLaw '26 main track! (archival and non-archival)!

did a little media criticism www.theverge.com/politics/777...

was just looking for @seantcollins.com’s “goofy at the crucification” post and google is so cool now

After 2 years in press, it's published! "Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain," is out in the 72nd volume of the Journal of the Copyright Society copyrightsociety.org/journal-entr... written with @katherinelee.bsky.social & @jtlg.bsky.social (2023)

The Bartz v. Anthropic settlement is the polar opposite of the Google Books settlement: a discrete one-time payment for past copying, on a discrete and closed-ended class, and making no attempt at all to deal with a larger forward-looking issues.

Here is the direct link to the paper: arxiv.org/abs/2505.12546

I’m excited to share that my paper with @jtlg.bsky.social , "The Files are in the Computer: On Copyright, Memorization, and Generative AI" (April 2024), is out in the AI Disrupting Law symposium issue of the Chicago-Kent Law Review! The full issue is here: scholarship.kentlaw.iit.edu/cklawreview/

The CFP for ACM CSLaw '26 is up! Deadline for main-track papers (archival and non-archival) is September 30! computersciencelaw.org/2026

I understand what the underlying probabilities mean, and therefore why this was worth giving a go. But I’m still occasionally like “How tf can someone extract entire books from a frontier company’s flagship LLM? Like we got _all_ of HP 1 with just ‘Mr. and Mrs. D’ as the seed prompt? What??”

Had a great time and learned a ton at ICML. But as an introvert, I’ve used up all my talking budget until the fall. Excited to get back to full time researchy things, and will hopefully have some exciting new results to share soon!

Strangers love to tell me “I can’t understand you, because of your MASK”. Dude, I am literally someone who gets paid to speak to large audiences while wearing a mask—I know I can be understood!

Happening now! Please swing by to talk about measurement!

Excited to be at #ICML '25! Please reach out if you'd like to chat. You can also find me presenting work at a few different spots, listed below!

Feeling so excited + grateful to be representing this paper at #ICML! Please stop by to talk about how to do more valid measurement for evaling gen AI systems! Work led by the incomparable @hannawallach.bsky.social and @azjacobs.bsky.social as a part of Microsoft’s AI and Society initiative!!

Some minor updates to our recent books memorization paper! I’ve separated out a new section 5 that I hope makes some of our ML findings about memorization clearer to a wider audience. Preprint here: arxiv.org/abs/2505.12546 1/8

"Llama 3.1 70B memorizes some books, like Harry Potter & the Sorcerer's Stone and 1984, almost entirely. ... HP is so memorized that, using a seed prompt consisting of just the first line of chapter 1, we can deterministically generate the entire book near-verbatim." papers.ssrn.com/sol3/papers....

Once again, I encourage folks speculating about what this means to read @pamelasamuelson.bsky.social on remedies. The range of possibilities is quite broad.

This opinion is a reminder that these cases are not general-purpose referenda on AI policy; they are hyper-technocratic copyright cases. Copyright draws lots of unsatisfying and counterintuitive distinctions, which is why you should hire and listen to copyright lawyers on the front end.