Join me and @mariusmosbach.bsky.social to chat about our work on frequency effects in unlearning — and how @ai2.bsky.social's Olmo helped us gain key insights. 💬 AMA: Tue, Oct 28 — 8:00 PT / 16:00 CEST 💡 Bring your questions! 🔗 discord.gg/ai2

We're presenting “Not all data are unlearned equally” at #COLM2025! We show that data properties shape how LLMs forget — stop by to chat more! 🗓 Wednesday, Oct 8 🕓 4:30–6:30 pm 📍 poster #710 (session 4) paper: arxiv.org/abs/2504.05058 Work with @mariusmosbach.bsky.social @sivareddyg.bsky.social

very happy to see the trend of a Behind the Scenes section catching on! transparent & honest science 👌 love the detailed montreal spots mentioned consider including such a section in your next appendix! (paper by @a-krishnan.bsky.social arxiv.org/pdf/2504.050...)

Our new paper in #PNAS (bit.ly/4fcWfma) presents a surprising finding—when words change meaning, older speakers rapidly adopt the new usage; inter-generational differences are often minor. w/ Michelle Yang, ‪@sivareddyg.bsky.social‬ , @msonderegger.bsky.social‬ and @dallascard.bsky.social‬👇(1/12)

📢 #SpeechTech & #SpeechScience researchers! We are thrilled to announce that Prof. Karen Livescu will keynote our Special Session on Interpretable Audio and Speech Models at #Interspeech2025: "What can interpretability do for us (and what can it not)?" 🗓️ Aug 18, 11:00 @interspeech.bsky.social

I am excited to announce that my paper "On the reliability of feature attribution methods for speech classification" has been accepted to #Interspeech2025! Co-authors: @hmohebbi.bsky.social, Arianna Bisazza, Afra Alishahi, @grzegorz.chrupala.me Find the preprint here: arxiv.org/abs/2505.16406

Come to my keynote tomorrow at the first official @queerinai.com workshop at #NAACL2025 to hear about how trans languaging is complex and cool, and how this makes it extra difficult to process computationally. I will have SO many juicy examples!

Chain-of-Thought (CoT) reasoning lets LLMs solve complex tasks, but long CoTs are expensive. How short can they be while still working? Our new ICML paper tackles this foundational question.

A must-read for anyone in NLP right now

Congratulations to Mila members @adadtur.bsky.social , Gaurav Kamath and @sivareddyg.bsky.social for their SAC award at NAACL! Check out Ada's talk in Session I: Oral/Poster 6. Paper: arxiv.org/abs/2502.05670

Incredibly proud of my students @adadtur.bsky.social and Gaurav Kamath for winning a SAC award at #NAACL2025 for their work on assessing how LLMs model constituent shifts.

💡 New ICLR paper! 💡 "On Linear Representations and Pretraining Data Frequency in Language Models": We provide an explanation for when & why linear representations form in large (or small) language models. Led by @jackmerullo.bsky.social, w/ @nlpnoah.bsky.social & @sarah-nlp.bsky.social

DeepSeek-R1 Thoughtology: Let’s <think> about LLM reasoning 142-page report diving into the reasoning chains of R1. It spans 9 unique axes: safety, world modeling, faithfulness, long context, etc. Now on arxiv: arxiv.org/abs/2504.07128

AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories We are releasing the first benchmark to evaluate how well automatic evaluators, such as LLM judges, can evaluate web agent trajectories.

Checkout Benno's notes about our impact of interpretability paper 👇. Also, we are organizing a workshop at #ICML2025 which is inspired by some of the questions discussed in the paper: actionable-interpretability.github.io

Check out our new paper on unlearning for LLMs 🤖. We show that *not all data are unlearned equally* and argue that future work on LLM unlearning should take properties of the data to be unlearned into account. This work was lead by my intern @a-krishnan.bsky.social 🔗: arxiv.org/abs/2504.05058

📢Excited to announce our upcoming workshop - Vision Language Models For All: Building Geo-Diverse and Culturally Aware Vision-Language Models (VLMs-4-All) @CVPR 2025! 🌐 sites.google.com/view/vlms4all

Agents like OpenAI Operator can solve complex computer tasks, but what happens when users use them to cause harm, e.g. spread misinformation? To find out, we introduce SafeArena (safearena.github.io), a benchmark to assess the capabilities of web agents to complete harmful web tasks. A thread 👇

📢 #SpeechTech & #SpeechScience researchers! ⏳ Reminder: The #Interspeech2025 deadline is approaching! 🚀 If your work focuses on interpretability in speech & audio, submit through our Special Session and showcase your research! 🎤 #Interpretability @interspeech.bsky.social