Great release from our engineering team! A lot of the major pain points have been addressed, and this is our first step towards supporting interpretability workflows on more realistic scenarios! Check it out!

Those of us who work in AI in the US today should take a moment to think today. Do not get distracted by the circus. Instead, let us pause to think carefully about our freedoms, our rights, and our responsibilities as citizens and professionals. It is a deadly serious moment.

I'm excited to share that this paper was accepted at ICLR 2026! We show that language models encode one of the most basic ingredients of a world model: the ability to distinguish plausible from implausible states. Check out the paper for more details! See you in Rio! Paper: arxiv.org/abs/2507.12553

In this amazing multidisciplinary collaboration, we report our early experience with the @openclaw-x.bsky.social ->

Are we all Agents of Chaos in AI? (Hope not!) In recent weeks using OpenClaw has taught us a lot about this wooly new kind of autonomous software agent. Its valuable to see what @NatalieShapira, @wendlerch et al. have seen: agentsofchaos.baulab.info/

Our research report on red-teaming stateful OpenClaw agents in the BauLab is finally out! 🥳 This awesome effort was led by @natalieshapira.bsky.social and involved 6 ClawBots and 20 researchers from various institutions. Check it out ➡️ agentsofchaos.baulab.info

You can now "pip install latentlens" 🔨 It comes with: * pre-computed embeddings for several popular LLMs and VLMs * a txt file with sentences describing WordNet concepts, which we recommend as a standard corpus to get embeddings from * ... Try it out and let us know what we can improve!

Our ICML 2025 workshop on Actionable Interpretability drew massive interest. But the same questions kept coming up: What does "actionable" mean? Is it achievable? How? We're ready to answer. 🧵

A totally normal morning in Cambridge, MA

How do you knock the induction heads out of an LM while preserving its ability to think? Is it even possible? @keremsahin22.bsky.social's work is worth reading if you haven't seen it yet. hapax.baulab.info

To accompany our paper, we also released a @hf.co space to explore agent trajectories and decoded cognitive maps from our eval outputs! Find it here: huggingface.co/spaces/proje...

Can LLMs figure out who you are from your anonymous posts? From a handful of comments, LLMs can infer where you live, what you do, and your interests; then search for you on the web. New 📄 w/ @SimonLermenAI, @joshua_swans, @AerniMichael, Nicholas Carlini, @florian_tramer 🧵

When we say an AI agent is “goal-directed”, what do we actually mean? In our new work, we study this question by combining behavioural and interpretability analysis in a language model agent navigating 2D grid worlds. Blog: projecttelos.substack.com/p/a-behaviou... Paper: arxiv.org/abs/2602.08964

Karpathy: here's a GPT in 5 lines of pure Python. That's all you need for AGI. Frontier labs: Bernie B2.5 implements GLGMHA (Great Latent Goated Maxi-Head Attention), scaling context size to 1 bazillion tokens

Opus 4.6 still tells me "Good Feedback". Phew, my job is safe (for the next ~3 months)

Surely a good omen, thanks Gemini

Had a bit of fun with @anthropic.com Claude artifacts tonight and ended up with two demos that feel pretty useful for language learners: an assistive reader that lets you export/practice new words, and a translation helper that explains your mistakes. Find them here: gsarti.com/langlearn

Does it matter how you prompt an LLM with a persona? Do LLMs respond differently to natural conversation history compared to names and explicit mentions? Go check out our new preprint! 👀

The Art of Wanting. About the question I see as central in AI ethics, interpretability, and safety. Can an AI take responsibility? I do not think so, but *not* because it's not smart enough. davidbau.com/archives/20...

Can models understand each other's reasoning? 🤔 When Model A explains its Chain-of-Thought (CoT) , do Models B, C, and D interpret it the same way? Our new preprint with @davidbau.bsky.social and @csinva.bsky.social explores CoT generalizability 🧵👇 (1/7)

Can you solve this algebra puzzle? 🧩 cb=c, ac=b, ab=? A small transformer can learn to solve problems like this! And since the letters don't have inherent meaning, this lets us study how context alone imparts meaning. Here's what we found:🧵⬇️

It was an honor to be part of this awesome project! Interpreto is a great up-and-coming tool for concept-based interpretability analyses of NLP models, check it out!

New year, new YouTube videos! We are resuming our regular interpretability seminar posts, with a fantastic talk by Deepti Ghadiyaram on interpreting diffusion models. Watch the video: youtu.be/4eqvABPX5rA

All interpretability research is either philosophy (affectionate) or stamp collecting (derogatory)

The NDIF ecosystem is growing! 🚀 nnterp will bridge the gap between fine-grained fiddling with model internals (nnsight) and low-code access to bespoke viz (workbench). Excited to work with @butanium.bsky.social and the @ndif-team.bsky.social to make it a standard in interp research!

Happy to announce I will be mentoring a SPAR project this Spring! ✨Check out the programme and apply by Jan 14th to work with me on understanding and mitigating implicit personalization in LLMs, i.e. how models form hidden beliefs about users that shape their responses.

This reads like a modern-day satirical adaptation of "The Lifecycle of Software Object" by Ted Chiang!

📣 I'm starting a postdoc at Northeastern University, where I will work on open-source NN interpretability with @davidbau.bsky.social and the @ndif-team.bsky.social. In 2026, we'll grow the NDIF ecosystem and democratize access to interpretability methods for academics and domain experts! 🚀

Our work on contrastive SAE steering for personalizing literary machine translation was accepted to EACL main! 🎉 Check it out! ⬇️

Here's my enormous round-up of everything we learned about LLMs in 2025 - the third in my annual series of reviews of the past twelve months simonwillison.net/2025/Dec/31/... This year it's divided into 26 sections! This is the table of contents:

Happy Holidays from NDIF! Our new NNsight version improves performance and enhances vLLM integration, including support for tensor parallelism.

I have been teaching myself to vibe code. Watch Claude Code grow my 780 lines to 13,600 - mandelbrot.page/coverage/ca... Two fundamental rules for staying in control: davidbau.com/archives/20...

Big news! 🗞️ I defended my PhD thesis "From Insights to Impact: Actionable Interpretability for Neural Machine Translation" @rug.nl @gronlp.bsky.social I'm grateful to my advisors @arianna-bis.bsky.social @malvinanissim.bsky.social and to everyone who played a role in this journey! 🎉 #PhDone

The CALAMITA (Challenging the Abilities of LAnguage Models in ITAlian) paper is now available on arXiv: arxiv.org/abs/2512.04759 We warmly thank all the individuals involved for their extraordinary work, dedication, and collaborative spirit that made this project possible!

Kinda crazy the improvement from Nano banana (left) to NB Pro (right): "Create an infographic explaining how model components contribute to the prediction process of a decoder-only Transformer LLM. Use the residual stream view of the Transformer by Elhage et al. (2021) in your presentation."

impactrank.org is an interesting take on how to rethink uni rankings to upweight quality rather than quantity. They use LLMs to extract "high impact" dependencies from papers and identify foundational work, tracing them back to PIs/unis by matching their DBLP entries. Have a look!

Humans and LLMs think fast and slow. Do SAEs recover slow concepts in LLMs? Not really. Our Temporal Feature Analyzer discovers contextual features in LLMs, that detect event boundaries, parse complex grammar, and represent ICL patterns.

New promising model for interpretability research just dropped!

Check out our awesome live-skeeted panel!

Follow @blackboxnlp.bsky.social for a live skeeting of the event!

Wrapping up my oral presentations today with our TACL paper "QE4PE: Quality Estimation for Human Post-editing" at the Interpretability morning session #EMNLP2025 (Room A104, 11:45 China time)! Paper: arxiv.org/abs/2503.03044 Slides/video/poster: underline.io/lecture/1315...

Presenting today our work "Unsupervised Word-level Quality Estimation Through the Lens of Annotator (Dis)agreement" at the Machine Translation morning session (Room A301, 11:45 China time). See you there! 🤗 Paper: aclanthology.org/2025.emnlp-m... Slides/video/poster: underline.io/events/502/s...

How can a language model find the veggies in a menu? New pre-print where we investigate the internal mechanisms of LLMs when filtering on a list of options. Spoiler: turns out LLMs use strategies surprisingly similar to functional programming (think "filter" from python)! 🧵

Language models can correctly answer questions about their previous intentions. www.anthropic.com/research/int...

Can AI simulate human behavior? 🧠 The promise is revolutionary for science & policy. But there’s a huge "IF": Do these simulations actually reflect reality? To find out, we introduce SimBench: The first large-scale benchmark for group-level social simulation. (1/9)

Our group @gronlp.bsky.social is coming in strong for #EMNLP2025! See you soon in Suzhou! 👋 🇨🇳

You can easily save up to 65% of compute while improving performance on reasoning tasks 🤯 👀 Meet EAGer: We show that monitoring token-level uncertainty lets LLMs allocate compute dynamically - spending MORE on hard problems, LESS on easy ones. 🧵👇

𝐃𝐨 𝐲𝐨𝐮 𝐫𝐞𝐚𝐥𝐥𝐲 𝐰𝐚𝐧𝐭 𝐭𝐨 𝐬𝐞𝐞 𝐰𝐡𝐚𝐭 𝐦𝐮𝐥𝐭𝐢𝐥𝐢𝐧𝐠𝐮𝐚𝐥 𝐞𝐟𝐟𝐨𝐫𝐭 𝐥𝐨𝐨𝐤𝐬 𝐥𝐢𝐤𝐞? 🇨🇳🇮🇩🇸🇪 Here’s the proof! 𝐁𝐚𝐛𝐲𝐁𝐚𝐛𝐞𝐥𝐋𝐌 is the first Multilingual Benchmark of Developmentally Plausible Training Data available for 45 languages to the NLP community 🎉 arxiv.org/abs/2510.10159

"Assuming linearly encoded concepts"

Very cool demonstration of how the @ndif-team.bsky.social Workbench allows for quick iteration on different prompt setups!