@afedercooper.bsky.social and I have a new paper that expands AI memorization tests to include near-verbatim copies, not just exact copies. We find that increases extraction/memorization of copyrighted content significantly. arxiv.org/abs/2603.24917

So glad to see this work (& other memorization works) being cited in Congressional testimony before the U.S. House Committee on Energy and Commerce. Dr. Jennifer King (Stanford HAI) talks about memorization of personal information & data privacy risks. More details: hai.stanford.edu/assets/files/t…

I gave a talk at the Google Privacy in ML Seminar last summer on privacy & memorization: "Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training". It's up on YouTube now if you're interested :) youtu.be/IzIsHFCqXGo?...

Some of these results really changed how i think about memorization. It’s not simply data. There are many factors (data properties, architecture, training objectives, capacity etc) that interplay to determine what exactly gets memorized.

Microsoft Research NYC is hiring a researcher in the space of AI and society!

Excited to share my work at Meta. Knowledge Distillation has been gaining traction for LLM utility. We find that distilled models don't just improve performance, they also memorize significantly less training data than standard fine-tuning (reducing memorization by >50%). 🧵

very cool work!!!

Syntax that spuriously correlates with safe domains can jailbreak LLMs - e.g. below with GPT4o mini Our paper (co w/ Vinith Suriyakumar) on syntax-domain spurious correlations will appear at #NeurIPS2025 as a ✨spotlight! + @marzyehghassemi.bsky.social, @byron.bsky.social, Levent Sagun

It is PhD application season again 🍂 For those looking to do a PhD in AI, these are some useful resources 🤖: 1. Examples of statements of purpose (SOPs) for computer science PhD programs: cs-sop.org [1/4]

"AI slop" seems to be everywhere, but what exactly makes text feel like "slop"? In our new work (w/ @tuhinchakr.bsky.social, Diego Garcia-Olano, @byron.bsky.social ) we provide a systematic attempt at measuring AI "slop" in text! arxiv.org/abs/2509.19163 🧵 (1/7)

After 2 years in press, it's published! "Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain," is out in the 72nd volume of the Journal of the Copyright Society copyrightsociety.org/journal-entr... written with @katherinelee.bsky.social & @jtlg.bsky.social (2023)

Excited to be attending ACL in Vienna next week! I’ll be co-presenting a poster with Niloofar Mireshghallah on our recent PII memorization work on July 29 16:00-17:30 Session 10 Hall 4/5 (& at LLM memorization workshop)! If you would like to chat memorization/privacy/safety/, please reach out :)

Privacy Ripple Effects has been accepted to the Findings of ACL 2025! 🎉 See you in Vienna! #ACL2025

Very excited to be joining Meta GenAI as a Visiting Researcher starting this June in New York City!🗽 I’ll be continuing my work on studying memorization and safety in language models. If you’re in NYC and would like to hang out, please message me :)

I am at CHI this week to present my poster (Framing Health Information: The Impact of Search Methods and Source Types on User Trust and Satisfaction in the Age of LLMs) on Wednesday April 30 CHI Program Link: programs.sigchi.org/chi/2025/pro... Looking forward to connecting with you all!

4/26 at 3pm: 'Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon' USVSN Sai Prashanth · @nsaphra.bsky.social et al Submission: openreview.net/forum?id=3E8...

Bummed to be missing ICLR, but if you’re interested in all things memorization, stop by poster #200 Hall 3 + Hall 2B on April 26 3-5:30 pm and chat with several of my awesome co-authors. We propose a taxonomy for different types of memorization in LMs. Paper: openreview.net/pdf?id=3E8YN...

[📄] Are LLMs mindless token-shifters, or do they build meaningful representations of language? We study how LLMs copy text in-context, and physically separate out two types of induction heads: token heads, which copy literal tokens, and concept heads, which copy word meanings.

BU and Boston are incredibly lucky to have Naomi!!!

Really liked this slide by @afedercooper.bsky.social on categorizing extraction vs regurgitation vs memorization of training data at CS&Law today!

This is some great work!! I personally feel that one of the bottlenecks with memorization evals is having access to the gigantic training data. Super cool to see we can still run reliable evals without having access to the training data!

Excited to be in Munich for my first ACM CS&Law! If you are interested in chatting about memorization + privacy/law in language models, we should hang out :)

If you're in the northeastern US and you're submitting a paper to COLM on March 27, you should absolutely be sending its abstract to New England NLP on March 28.

*Please repost* @sjgreenwood.bsky.social and I just launched a new personalized feed (*please pin*) that we hope will become a "must use" for #academicsky. The feed shows posts about papers filtered by *your* follower network. It's become my default Bluesky experience bsky.app/profile/pape...

I'm searching for some comp/ling experts to provide a precise definition of “slop” as it refers to text (see: corp.oup.com/word-of-the-...) I put together a google form that should take no longer than 10 minutes to complete: forms.gle/oWxsCScW3dJU... If you can help, I'd appreciate your input! 🙏

We’ve been receiving a bunch of questions about a CFP for GenLaw 2025. We wanted to let you know that we chose not to submit a workshop proposal this year (we need a break!!). We’ll be at ICML though and look forward to catching up there! You can watch our prior videos!

Nicholas is leaving GDM at the end of this week, and we're feeling big sad about it: nicholas.carlini.com/writing/2025...

Last CFP at ACM CS&Law ‘25! Please submit your two-minute lightning talks. It’s a great way to advertise work to the community and to find potential new collaborators! More info (including about registration) on the website: computersciencelaw.org/2025

What happens if we fine-tune an LLM on more PII? We find that PII that wasn’t previously extracted gets extracted after fine-tuning on *other* PII. This could have implications for earlier seen data (e.g. during post-training or further fine-tuning). 🧵 paper: arxiv.org/pdf/2502.15680

More on LLM training dynamics from @jaydeepborkar.bsky.social: Even when personal information is sampled independently from the rest of the training data, interactions in term statistics can still increase leakage. arxiv.org/abs/2502.15680

A antique sci-fi treat for everyone talking about language models, copyright, plagiarism, and when humans can exhaust the space of all possible art.