Bill Ackman gotta be on the third draft of a tweet longer than Middlemarch right now

I’m kinda known as a copyright person, but (even in memorization) I mainly study how to draw reliable conclusions from large-scale AI/ML systems. There’s a long spiel why, but today I feel defeated. 100 hours/week on this for 6 years, just to find out a parent treats Gemini in search as ground-truth

The NeurIPS position track didn't take a large number of extraordinary papers that surpassed the acceptance bar, limiting the acceptance rate to an unusually low 6%. If you have a rejected paper at the intersection of ML and law, consider submitting to ACM CSLaw '26.

Our paper "Machine Unlearning Doesn't Do What You Think" was accepted for presentation at NeurIPS Congrats @afedercooper.bsky.social and @katherinelee.bsky.social, who led the effort arxiv.org/abs/2412.06966

One more week to submit to CSLaw '26!!

For an update on the state of play in the generative AI copyright cases, try this podcast: shows.acast.com/arbiters-of-...

15 days left to submit to the CSLaw '26 main track! (archival and non-archival)!

did a little media criticism www.theverge.com/politics/777...

was just looking for @seantcollins.com’s “goofy at the crucification” post and google is so cool now

After 2 years in press, it's published! "Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain," is out in the 72nd volume of the Journal of the Copyright Society copyrightsociety.org/journal-entr... written with @katherinelee.bsky.social & @jtlg.bsky.social (2023)

The Bartz v. Anthropic settlement is the polar opposite of the Google Books settlement: a discrete one-time payment for past copying, on a discrete and closed-ended class, and making no attempt at all to deal with a larger forward-looking issues.

Here is the direct link to the paper: arxiv.org/abs/2505.12546

I’m excited to share that my paper with @jtlg.bsky.social , "The Files are in the Computer: On Copyright, Memorization, and Generative AI" (April 2024), is out in the AI Disrupting Law symposium issue of the Chicago-Kent Law Review! The full issue is here: scholarship.kentlaw.iit.edu/cklawreview/

The CFP for ACM CSLaw '26 is up! Deadline for main-track papers (archival and non-archival) is September 30! computersciencelaw.org/2026

I understand what the underlying probabilities mean, and therefore why this was worth giving a go. But I’m still occasionally like “How tf can someone extract entire books from a frontier company’s flagship LLM? Like we got _all_ of HP 1 with just ‘Mr. and Mrs. D’ as the seed prompt? What??”

Had a great time and learned a ton at ICML. But as an introvert, I’ve used up all my talking budget until the fall. Excited to get back to full time researchy things, and will hopefully have some exciting new results to share soon!

Strangers love to tell me “I can’t understand you, because of your MASK”. Dude, I am literally someone who gets paid to speak to large audiences while wearing a mask—I know I can be understood!

Happening now! Please swing by to talk about measurement!

Excited to be at #ICML '25! Please reach out if you'd like to chat. You can also find me presenting work at a few different spots, listed below!

Feeling so excited + grateful to be representing this paper at #ICML! Please stop by to talk about how to do more valid measurement for evaling gen AI systems! Work led by the incomparable @hannawallach.bsky.social and @azjacobs.bsky.social as a part of Microsoft’s AI and Society initiative!!

Some minor updates to our recent books memorization paper! I’ve separated out a new section 5 that I hope makes some of our ML findings about memorization clearer to a wider audience. Preprint here: arxiv.org/abs/2505.12546 1/8

"Llama 3.1 70B memorizes some books, like Harry Potter & the Sorcerer's Stone and 1984, almost entirely. ... HP is so memorized that, using a seed prompt consisting of just the first line of chapter 1, we can deterministically generate the entire book near-verbatim." papers.ssrn.com/sol3/papers....

“these are hypertechnocratic” is one of the most important things you can draw from this morning’s ruling. In other words, hesitate before drawing parallels between this case and your most (loved|hated) AI training use case. (@chup.blakereid.org’s whole thread is great)

very proud of my hometown. stunned, but proud.

idk folks, have read today’s decision by Alsup and I think it’s narrower than news headlines are indicating (shocker).To me this reads like (mostly) a win for Anthropic, but in this specific case with respect to these claims from these specific plaintiffs.

The conservative movement is lying about basically every empirical "debate" of our time, from climate change to voter fraud to vaccines. Why is it so impossible for reactionary centrists to contemplate the possibility they're lying about trans healthcare too?

Ars Technica has a great writeup of my new study with @afedercooper.bsky.social and Amy Cyphert on memorization in AI language models arstechnica.com/ai/2025/06/s...

I was wondering when tech media was going to pick up this paper. :) "Study: Meta AI model can reproduce almost half of Harry Potter book": arstechnica.com/features/202... Coincidentally, I made a video about this paper a couple of weeks ago in a response to a comment I got elsewhere online.

For AI industry critics, the big takeaway here is that—at least for some models and some books—memorization is not a fringe phenomenon.

uh...lol(?) at all the internet commenters who think that we/the authors on our recent books extraction paper are tacitly endorsing or supporting Rowling because Harry Potter is one of the books we tested (that happens to be highly memorized)

Today's Skrmetti opinion is devastating to transgender children and families who live in states with cruel laws barring gender affirming care. But it is very important to recognize this opinion does not give private entities, legislatures, or the President carte blanche to discriminate! 1/x

excited to see that folks are finding their way to our books memorization project. mashable.com/article/meta... I’ll also be giving a talk on this work at the ICML memorization workshop in July. Please check out the paper for more details! arxiv.org/abs/2505.12546 cc @marklemley.bsky.social

Researchers find Llama 3.1 recalls large parts of popular copyrighted books, possibly weakening AI industry claims that such memorization is fringe behavior (Timothy B. Lee/Understanding AI) Main Link | Techmeme Permalink

Gavin Newsom vetoed a data broker privacy bill that our legislature passed last year that would have allowed Californians to opt out of all data brokers with a single click.

Tim Lee wrote an awesome short piece on my recent project with @marklemley.bsky.social and others on extracting memorized books from LLMs. Please check it out for a very clear explanation of what we did, and why the results have important consequences! www.understandingai.org/p/metas-llam...

Licensing fee demand incoming

wow, a research career milestone: someone angrily saying my paper has been done before…by pointing to my prior work that does something different (and which this current work builds on / depends on)

@marklemley.bsky.social and @cjsprigman.bsky.social are doing some really important things—and winning. Beyond the general public, I hope other academics are taking note. There’s a lot (that can take many forms, public and private) each of us can do right now.

I really enjoyed watching Andor, but I absolutely don’t enjoy living in Andor

oh to be a seal in a tide pool, instead of a grouch on the internet

This is my entire AI policy position in a nutshell

Look, I don’t actually have a lot of strong opinions about AI policy, but one of them is that people who do have strong opinions should take even just a few minutes to think through the obvious implications of the claims they make.

Can you train a performant language model using only openly licensed text? We are thrilled to announce the Common Pile v0.1, an 8TB dataset of openly licensed and public domain text. We train 7B models for 1T and 2T tokens and match the performance similar models like LLaMA 1 & 2

well that's the fastest a paper of mine has been read/cited. thanks @lucy3.bsky.social. and awesome work on LDA x LLMs!

i swam like half a mile today, and will be sleeping for the rest of this vacation

I briefly noted this very interesting paper on memorization by LLMs when it came out, but it just showed up in my SSRN abstracts and that reminded me to write about it in a bit more detail. 🧵 arxiv.org/abs/2505.12546

first full day of vacation is going great. made it until 10:30am before writing a line of code. hopefully tomorrow i can extend that to at least noon.

AI benchmarking culture is completely out of control. Tables with dozens of methods, datasets, and bold numbers, trying to answer a question that perhaps no one should be asking anymore.