If I had to summarize this paper in one philosophical question: Why do we call some things by the same name and others by different names? Related to old philosophy of logic/language is Quine's work (1960): en.wikipedia.org/wiki/Inscrut...

My first last-author paper is out! If you saw this dog below and someone showed you the second image, would you consider them the same word/concept? (more examples in Ada's thread) We study if VLMs agree with humans on this and revisit old questions around shape vs. texture bias in vision

I'll be presenting OvertonBench at #ICLR2026 in Rio later this week! 📍Sat, Apr 25, 10:30am in Pavilion 4 (#4109) Please DM me if you'd like to chat about pluralistic / value alignment, societal impacts, epistemology, fairness, evals, etc

I'll be at ICLR! Will present our LatentLens paper at the Re-Align workshop and happy to chat (ideally at the beach 🏖️)

What are your favorite papers that can serve as excellent examples how to write great scientific paper, present results, great figures, make it engaging and easy to follow? Doesn't necessarily have to be the most cited or impactful ones

Inspired by @bennokrojer.bsky.social, we included a Behind the Scenes section 🎬 The goal is to make science more transparent 🔍, share lessons learned 🧠, and provide a more realistic lens on the research journey 👣 8/ bsky.app/profile/benn...

🚨New Paper!🚨 How do reasoning LLMs handle inferences that have no deterministic answer? We find that they diverge from humans in some significant ways, and fail to reflect human uncertainty… 🧵(1/10)

People often say (myself too): Interpretability on AI is so much easier than neuroscience! We can inspect everything and even retrain (vs carefully poke a little into the brain)! One big advantage in neuroscience I often forget: We're quite literally *inside* the thing we're studying

You can now "pip install latentlens" 🔨 It comes with: * pre-computed embeddings for several popular LLMs and VLMs * a txt file with sentences describing WordNet concepts, which we recommend as a standard corpus to get embeddings from * ... Try it out and let us know what we can improve!

Finally getting into this classic Let's see if by the end I'll have a clearer idea what type of science some fields of AI are, like interpretability What are our paradigms?

Google decided to show this as my first sentence from my website (and not any of the sentences actually at the top of the website)

What does it mean for visual tokens to be "interpretable" to LLM? And how to we measure it? These, and many more pressing questions are addressed! Introducing LatentLens -- a new, more faithful tool for interpretability! Honoured to have collaborated with @bennokrojer.bsky.social on this!

For every one of my papers, I try to include a "Behind the Scenes" section I think this paper in particular has a lot going on behind the scenes; from lessons learned to personal reflections let me share some

This will be my last paper of the phd, can't believe it's been almost 5 years! It is the work i am most proud of and believe has the most potential. Feels right to wrap it up with this one

🚨New paper Are visual tokens going into an LLM interpretable 🤔 Existing methods (e.g. logit lens) and assumptions would lead you to think “not much”... We propose LatentLens and show that most visual tokens are interpretable across *all* layers 💡 Details 🧵

The visual world is composed of objects, and those objects are composed of features. But do VLMs exploit this compositional structure when processing multi-object scenes? In our 🆒🆕 #ICLR2026 paper, we find they do – via emergent symbolic mechanisms for visual binding. 🧵👇

📢 I am hiring a highly-motivated Ph.D student at the University of Copenhagen to work on tokenization-free NLP. Read our previous work in this topic: aclanthology.org/2025.emnlp-m... aclanthology.org/2023.emnlp-m... openreview.net/forum?id=FkS... Apply by March 8: employment.ku.dk/phd/?show=1563

nooo claude's --verbose is gone/broken, how will i now catch wrong assumptions before it goes on for 10 minutes?

A paper that should get more attention, for those interested in building truly multimodal models (aka not just plugging stuff post hoc into LLMs): arxiv.org/abs/2412.06646 TLDR: Counterintuitively, native multimodal modes seem *less* unified internally (interp)

Epstein’s economic power among academics was made possible by a capitalist system that makes higher education dependent on the charity economy rather than a public good supported by taxing the rich

🎉 Excited to share our new paper which was accepted to #AAAI2026! As LLMs become increasingly used as sources of factual knowledge, we ask: Do they perform equitably across users of different backgrounds? 🧵⬇️ 1/6

Figuring out the right rules+practices for my CLAUDE.md file and generally how to prompt Claude Code, has taught me more about software/ML engineering paradigms/concepts/best-practices than anything I've learned in uni

Listenining to Michelle Obama's audiobook "Becoming" (loving it btw) in 2026 is wild, an almost comical contrast to now... What hopeful times it was back then, as I'm now at the chapters (2006-2008) describing their 2008 run for president

Wild output from Gemini i got two weeks ago i wonder if the appearance on tv is meant to represent the sin of pride

With the latest coding agents there's almost no excuse anymore to publish a paper without any cool demo or interactive data explorer For my current paper it made it so much easier to quickly grasp different interp tools and their effects throughout the whole project

New blog post 📜 "Better late than never: Getting into interpretability in 2025" It's been a great year pivoting into interp and i wanted to reflect on it bennokrojer.com/interp.html

Really enjoyed the discussions in the UT Austin NLP group!

Couldn't have wished for a better place to do my PhD, come apply!

I'll be at COLM! Excited to chat about about anything vision+language, interpretability, cogsci/psych, embedding spaces, visual reasoning, video/world models

Devoured this book in 18 hours, usually not a big fan of audio books! It covered lots from crowdworker rights, the ideologies (doomers, EA, ...) and the silicon valley startup world to the many big egos and company-internal battles Great work by @karenhao.bsky.social

Lmao

Congratulations @bennokrojer.bsky.social on passing your PhD proposal exam! A great presentation and exciting work!

very happy to see the trend of a Behind the Scenes section catching on! transparent & honest science 👌 love the detailed montreal spots mentioned consider including such a section in your next appendix! (paper by @a-krishnan.bsky.social arxiv.org/pdf/2504.050...)

Super cool work on quantifying with NLP how language evolves through generations In linguistics, the "apparent time hypothesis" famously discusses this but never empirically tests it

I miss Edinburgh and its wonderful people already!! Thanks to @tallinzen.bsky.social and @edoardo-ponti.bsky.social for inspiring discussions during the viva! I'm now exchanging Arthur's Seat for Mont Royal to join @sivareddyg.bsky.social's wonderful lab @mila-quebec.bsky.social 🤩

Started a new podcast with @tomvergara.bsky.social ! Behind the Research of AI: We look behind the scenes, beyond the polished papers 🧐🧪 If this sounds fun, check out our first "official" episode with the awesome Gauthier Gidel from @mila-quebec.bsky.social : open.spotify.com/episode/7oTc...

Turns out condensing your research into 3min is very hard but also teaches you a lot Finally the video from Mila's speed science competition is on YouTube! From a soup of raw pixels to abstract meaning t.co/RDpu1kR7jM

Excited to share the results of my recent internship! We ask 🤔 What subtle shortcuts are VideoLLMs taking on spatio-temporal questions? And how can we instead curate shortcut-robust examples at a large-scale? We release: MVPBench Details 👇🔬

Top 99% in Boston 💪 Love these interactive maps

When you forget your leftover rice in the fridge for 3 weeks

Maybe it's just that I'm now paying more attention to the good parts again but since this post bluesky seems more fun again. Still not many paper discussions going on but saw some fun general posts

It was tough just logging back into my retired twitter account and to see a timeline that is so much fuller with interesting research discourse... And yes I've tried to customizing my feeds and whatnot but no feed can fix a lack of posts

I finish my work day with the conclusion that code assistants are maybe net negative for my short-term progress and most likely negative for my long-term progress and learning Also sycophancy is annoying af

Attend my AI 2025 bootcamp

This pattern is going to repeat in one domain after another, and gradually force us to admit that 60% of every job is networking, knowing who to trust, and doing poorly justified risk/benefit assessment.

with great responsibility comes a great amount of reimbursements to file

are there any analysis/interp papers on image editing? papers that are more insights than performance

Day 13: (the original mega-thread has become too long and nested so reposting now as a new strategy) Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models A few notes below 👇 I took less digital notes this time as i was sitting outside in the sun reading 🌞

This is one of the most-shared posts on Bluesky in the past day and it's just completely false. You might think ChatGPT is a *bad* search engine, or prefer another search engine. But it has had integrated web search since last year.

A must-read for anyone in NLP right now