Delighted Sasha's (first year PhD!) work using mech interp to study complex syntax constructions won an Outstanding Paper Award at EMNLP! Also delighted the ACL community continues to recognize unabashedly linguistic topics like filler-gaps... and the huge potential for LMs to inform such topics!

Interested in doing a PhD at the intersection of human and machine cognition? ✨ I'm recruiting students for Fall 2026! ✨ Topics of interest include pragmatics, metacognition, reasoning, & interpretability (in humans and AI). Check out JHU's mentoring program (due 11/15) for help with your SoP 👇

🧠 New at #NeurIPS2025! 🎵 We're far from the shallow now🎵 TL;DR: We introduce the first "reasoning embedding" and uncover its unique spatio-temporal pattern in the brain. 🔗 arxiv.org/abs/2510.228...

Introducing Global PIQA, a new multilingual benchmark for 100+ languages. This benchmark is the outcome of this year’s MRL shared task, in collaboration with 300+ researchers from 65 countries. This dataset evaluates physical commonsense reasoning in culturally relevant contexts.

Very excited to be going to Chicago for @agnescallard.bsky.social's famous Night Owls next week! I'll be discussing my essay "ChatGPT and the Meaning of Life". Hope to see you there if you're local!

If I spill the tea—“Did you know Sue, Max’s gf, was a tennis champ?”—but then if you reply “They’re dating?!” I’d be a bit puzzled, since that’s not the main point! Humans can track what’s ‘at issue’ in conversation. How sensitive are LMs to this distinction? New paper w/ @sangheekim.bsky.social!

I will be recruiting PhD students via Georgetown Linguistics this application cycle! Come join us in the PICoL (pronounced “pickle”) lab. We focus on psycholinguistics and cognitive modeling using LLMs. See the linked flyer for more details: bit.ly/3L3vcyA

"Although I hate leafy vegetables, I prefer daxes to blickets." Can you tell if daxes are leafy vegetables? LM's can't seem to! 📷 We investigate if LMs capture these inferences from connectives when they cannot rely on world knowledge. New paper w/ Daniel, Will, @jessyjli.bsky.social

Honored to get the chance to contribute to the Chinese dataset! And had a great time working with all the awesome collaborators!

Excited to present this at COLM tomorrow! (Tuesday, 11:00 AM poster session)

I will be giving a short talk on this work at the COLM Interplay workshop on Friday (also to appear at EMNLP)! Will be in Montreal all week and excited to chat about LM interpretability + its interaction with human cognition and ling theory.

Traveling to my first @colmweb.org🍁 Not presenting anything but here are two posters you should visit: 1. @qyao.bsky.social on Controlled rearing for direct and indirect evidence for datives (w/ me, @weissweiler.bsky.social and @kmahowald.bsky.social), W morning Paper: arxiv.org/abs/2503.20850

On my way to #COLM2025 🍁 Check out jessyli.com/colm2025 QUDsim: Discourse templates in LLM stories arxiv.org/abs/2504.09373 EvalAgent: retrieval-based eval targeting implicit criteria arxiv.org/abs/2504.15219 RoboInstruct: code generation for robotics with simulators arxiv.org/abs/2405.20179

I’m at #COLM2025 from Wed with: @siyuansong.bsky.social Tue am introspection arxiv.org/abs/2503.07513 @qyao.bsky.social Wed am controlled rearing: arxiv.org/abs/2503.20850 @sashaboguraev.bsky.social INTERPLAY ling interp: arxiv.org/abs/2505.16002 I’ll talk at INTERPLAY too. Come say hi!

Heading to #COLM2025 to present my first paper w/ @jennhu.bsky.social @kmahowald.bsky.social ! When: Tuesday, 11 AM – 1 PM Where: Poster #75 Happy to chat about my work and topics in computational linguistics & cogsci! Also, I'm on the PhD application journey this cycle! Paper info 👇:

🤖 🧠 NEW BLOG POST 🧠 🤖 What skills do you need to be a successful researcher? The list seems long: collaborating, writing, presenting, reviewing, etc But I argue that many of these skills can be unified under a single overarching ability: theory of mind rtmccoy.com/posts/theory...

The compling group at UT Austin (sites.utexas.edu/compling/) is looking for PhD students! Come join me, @kmahowald.bsky.social, and @jessyjli.bsky.social as we tackle interesting research questions at the intersection of ling, cogsci, and ai! Some topics I am particularly interested in:

Can AI aid scientists amidst their own workflows, when they do not know step-by-step workflows and may not know, in advance, the kinds of scientific utility a visualization would bring? Check out @sebajoe.bsky.social’s feature on ✨AstroVisBench:

Simon Goldstein and I have a new paper, “What does ChatGPT want? An interpretationist guide”. The paper argues for three main claims. philpapers.org/rec/GOLWDC-2 1/7

I did a QA with Quanta about interpretability and training dynamics! I got to talk about a bunch of research hobby horses and how I got into them.

Why does AI sometimes fail to generalize, and what might help? In a new paper (arxiv.org/abs/2509.16189), we highlight the latent learning gap — which unifies findings from language modeling to agent navigation — and suggest that episodic memory complements parametric learning to bridge it. Thread:

Announcing the first (and perhaps only) Multilingual Minds and Machines Meeting! Come join us in Nijmegen, June 22-23, 2026, if you are interested in computational models of human multilingualism: mmmm2026.github.io

Did you know? ❌77% of language models on @hf.co are not tagged for any language 📈For 95% of languages, most models are multilingual 🚨88% of models with tags are trained on English In a new blog post, @tylerachang.bsky.social and I dig into these trends and why they matter! 👇

Our new lab for Human & Machine Intelligence is officially open at Princeton University! Consider applying for a PhD or Postdoc position, either through Computer Science or Psychology. You can register interest on our new website lake-lab.github.io (1/2)

COLM 2025 accepted submissions are now public: openreview.net/group?id=col... Congratulations to all the authors, and see you all in Montreal!

Can AI introspect? Surprisingly tricky to define what that means! And also interesting to test. New work from @siyuansong.bsky.social, @harveylederman.bsky.social, @jennhu.bsky.social and me on introspection in LLMs. See paper and thread for a definition and some experiments!

Can AI models introspect? What does introspection even mean for AI? We revisit a recent proposal by Comșa & Shanahan, and provide new experiments + an alternate definition of introspection. Check out this new work w/ @siyuansong.bsky.social, @harveylederman.bsky.social, & @kmahowald.bsky.social 👇

exciting new paper from Siyuan! I really enjoyed working with him on this, inspired by important work by Murray Shanahan and Julia Comsa. Hard questions about how to operationalize the notion of “introspection” that’s relevant for practical applications in AI today. Hope you’ll check it out!

How reliable is what an AI says about itself? The answer depends on whether models can introspect. But, if an LLM says its temperature parameter is high (and it is!)….does that mean it’s introspecting? Surprisingly tricky to pin down. Our paper: arxiv.org/abs/2508.14802 (1/n)

A dataset of ancient Chinese writings to study with LLMs, including 170K sentences for pretraining With 10K words, mapping to modern word (when applicable) There are so many fascinating questions out there www.arxiv.org/abs/2508.15791

This Friday NEMI 2025 is at Northeastern in Boston, 8 talks, 24 roundtables, 90 posters; 200+ attendees. Thanks to goodfire.ai/ for sponsoring! nemiconf.github.io/summer25/ If you can't make it in person, the livestream will be here: www.youtube.com/live/4BJBis...

Call for papers for Evolang 2026 is now posted! evolang2026.org. Workshop proposals due Sept 22nd. Paper submissions October 26th. Evolang is interdisciplinarity at its best. We hope to see you in Plovdiv!!

In neuroscience, we often try to understand systems by analyzing their representations — using tools like regression or RSA. But are these analyses biased towards discovering a subset of what a system represents? If you're interested in this question, check out our new commentary! Thread:

Is the Language of Thought == Language? A Thread 🧵 New Preprint (link: tinyurl.com/LangLOT) with @alexanderfung.bsky.social, Paris Jaggers, Jason Chen, Josh Rule, Yael Benn, @joshtenenbaum.bsky.social, ‪@spiantado.bsky.social‬, Rosemary Varley, @evfedorenko.bsky.social 1/8

1/7 If you're at CogSci 2025, I'd love to see you at my talk on Friday 1pm PDT in Nob Hill A! I'll be talking about our work towards an implemented computational model of noisy-channel comprehension (with @postylem.bsky.social, Ted Gibson, and @rplevy.bsky.social).

Looking forward to attending #cogsci2025 (Jul 29 - Aug 3)! I’m especially excited to meet students who will be applying to PhD programs in Computational Ling/CogSci in the coming cycle. Please reach out if you want to meet up and chat! Email is the best way, but DM also works if you must! quick🧵:

Heading to #cogsci2025 this week! I'm interested in language, learning, and computational modeling. I’ll be around from July 29 morning to August 3 evening—excited to chat and get advice as someone just starting out in research! 😃

Heading to CogSci this week! ✈️ Find me giving talks on: 💬 Prod-comp asymmetry in children and LMs (Thu 7/31) 💬 How people make sense of nonsense (Sat 8/2) 📣 Also, I’m recruiting grad students + postdocs for my new lab at Hopkins! 📣 If you’re interested in language / cognition / AI, let’s chat! 😄

Next week I’ll be in Vienna for my first *ACL conference! 🇦🇹✨ I will present our new BLiMP-NL dataset for evaluating language models on Dutch syntactic minimal pairs and human acceptability judgments ⬇️ 🗓️ Tuesday, July 29th, 16:00-17:30, Hall X4 / X5 (Austria Center Vienna)

new homepage😀: siyuansong.site

Very excited for this workshop!

Excited to announce the first workshop on CogInterp: Interpreting Cognition in Deep Learning Models @ NeurIPS 2025! 📣 How can we interpret the algorithms and representations underlying complex behavior in deep learning models? 🌐 coginterp.github.io/neurips2025/ 1/4

NOW AVAILABLE! Proceedings of the LSA Vol. 10. The 89 articles in the volume stem from rigorous research presented at the 2025 LSA Annual Meeting & provide a marvelous showcase of the breadth & vitality of linguistics today, spanning the linguistic spectrum. tinyurl.com/yxryppk9 #Proceedings #LSA

🚨 New preprint! 🚨 Everyone loves causal interp. It’s coherently defined! It makes testable predictions about mechanistic interventions! But what if we had a different objective: predicting model behavior not under mechanistic interventions, but on unseen input data?

Experimentology is out today!!! A group of us wrote a free online textbook for experimental methods, available at experimentology.io - the idea was to integrate open science into all aspects of the experimental workflow from planning to design, analysis, and writing.

The New England Mechanistic Interpretability Workshop, NEMI 2025 is August 22 in Boston. Talks, posters, meals, discussion... Most of all, an excellent chance to chat about new ideas with other great researchers in the field! Help spread the word - register and repost - bsky.app/profile/koy...

How can we better understand how models make predictions and which components of a training dataset are shaping their behaviors? In April we introduced OLMoTrace, a feature that lets you trace the outputs of language models back to their full training data in real time. 🧵

#CoreCognition #LLM #multimodal #GrowAI We spent 3 years to curate 1503 classic experiments spanning 12 core concepts in human cognitive development and evaluated on 230 MLLMs with 11 different prompts for 5 times to get over 3.8 millions inference data points. A thread (1/n) - #ICML2025 ✅

As part of the workshop, we are also organizing a shared task to develop a collaborative physical commonsense reasoning evaluation dataset. See the shared task page for more information: sigtyp.github.io/st2025-mrl.h....

When it comes to text prediction, where does one LM outperform another? If you've ever worked on LM evals, you know this question is a lot more complex than it seems. In our new #acl2025 paper, we developed a method to find fine-grained differences between LMs: 🧵1/9