"Hello world!" sounds like a word followed by a crossword clue for that word: "Hell = Low world"

A linguistic note about David Copperfield and Demon Copperhead 🧵 [very minor spoilers for both] 1/n

New position paper! We argue that symbolic and neural network models are not in opposition to each other, but occupy different levels of analysis, and also outline a new research agenda for better understanding the relationship between them. Please check out the paper / thread for more details!

🤖 🧠 NEW PAPER ON COGSCI & AI 🧠 🤖 Recent neural networks capture properties long thought to require symbols: compositionality, productivity, rapid learning So what role should symbols play in theories of the mind? For our answer...read on! Paper: arxiv.org/abs/2508.05776 1/n

According to Jane Austen, linguists are extraordinarily cold-hearted. (Though at least we're not as bad as mathematicians!)

So much research is being done about LLMs that it's hard to stay on top of the literature. To help with this, I've made a list of all the most important papers from the past 8 years: rtmccoy.com/pubs/ I hope you enjoy!

July the 4th be with you!

New paper: "Large Language Models and Emergence: A Complex Systems Perspective" (D. Krakauer, J. Krakauer, M. Mitchell). We look at claims of "emergent capabilities" & "emergent intelligence" in LLMs from the perspective of what emergence means in complexity science. arxiv.org/pdf/2506.11135

The word "laundry" contains both steps of the laundry process: 1. Undry 2. Dry

Happy to announce the first workshop on Pragmatic Reasoning in Language Models — PragLM @ COLM 2025! 🎉 How do LLMs engage in pragmatic reasoning, and what core pragmatic capacities remain beyond their reach? 🌐 sites.google.com/berkeley.edu/praglm/ 📅 Submit by June 23rd

Had a fun visit to UChicago/TTIC over the past couple days - really great group doing NLP/CompLing there!

🤖🧠 Paper out in Nature Communications! 🧠🤖 Bayesian models can learn rapidly. Neural networks can handle messy, naturalistic data. How can we combine these strengths? Our answer: Use meta-learning to distill Bayesian priors into a neural network! www.nature.com/articles/s41... 1/n

At MIT for the day to speak at the NLP seminar! Say hi if you're around and need a break from NeurIPS drafting!

I constructed today's NYT crossword! This one has some personal connections, described at the WordPlay article by @samcorbin.bsky.social (contains spoilers): www.nytimes.com/2025/05/06/c... I hope you enjoy!

🧩🦾New paper🦾🧩 We use Linguistics Olympiad puzzles to test Large Language Models Check it out if you're at NAACL! It's being presented by Nathan Chi at 4:00 today in the LoResMT workshop aclanthology.org/2025.loresmt...

Made a new assignment for a class on Computational Psycholinguistics: - I trained a Transformer language model on sentences sampled from a PCFG - The students' task: Given the Transformer, try to infer the PCFG (w/ a leaderboard for who got closest) Would recommend! 1/n

The past few years have been a wild time to work in NLP - huge amounts of progress, hype, uncertainty, and excitement. This piece does a fantastic job of documenting what that's been like. I had fun being one of the interviewees! By John Pavlus in @quantamagazine.bsky.social

The initial application deadline is coming up (April 30 - but we will continue to review applications until the position is filled, so it's worth applying even if a little after that). Please share with any computational linguists and/or syntacticians who may be interested!

The Department of Linguistics at Yale University is seeking to hire a one-year lecturer with expertise in Computational Linguistics and/or Syntax for the 2025-2026 school year. See the job ad for more info: linguistlist.org/issues/36/11... Please send to any who might be interested!

I'm excited to be visiting UPenn tomorrow to speak at the ILST Seminar! Come by if you'll be around! web.sas.upenn.edu/lcs/news-eve...

Cognitive scientists who work across levels of analysis should be called Marr's Rovers

🤖🧠New paper!🧠🤖 A common assumption: In neural networks, "inductive bias" = "model architecture" But in fact initial weights also greatly influence inductive bias! This makes it possible for the same bias to be realized in very different architectures - albeit w/ limits arxiv.org/abs/2502.20237

Excited to announce that the (stellar!) program for NASSLLI is out now: nasslli25.shane.st Further details on registration and whatnot to follow in the coming weeks. Please do circulate to anyone you know who may be interested in attending!

The "ash heap of history" is where we store all the æ's that are no longer used in English spelling

A skill for academics that might become valuable in the coming years: LLM SEO. That is, writing your papers in a way that will get Deep Research to like them (I hope that this won't actually matter....but I wouldn't be surprised if it does. Could really influence the reach your work has.)

LMs need linguistics! New paper, with @futrell.bsky.social, on LMs and linguistics that conveys our excitement about what the present moment means for linguistics and what linguistics can do for LMs. Paper: arxiv.org/abs/2501.17047. 🧵below.

I added a new assignment to my Computational Linguistics class last semester: - Choose a linguistic phenomenon in a language other than English - Give a 3-minute presentation about that phenomenon & how it would pose a challenge for computational models Would recommend! 1/n

🔥While LLM reasoning is on people's minds... Here's a shameless plug for our work comparing o1 to previous LLMs (extending "Embers of Autoregression"): arxiv.org/abs/2410.01792 - o1 shows big improvements over GPT-4 - But qualitatively it is still sensitive to probability 1/4

My commentary on the do's and don'ts of cognitive evaluations in LLMs is now out in Nature Human Behavior: doi.org/10.1038/s415... posting here with a figure that didn't make it into the final draft and is now instead a boring table :P #CogSci #LLMs #AI

🪐 🚀 Physics 🚀 🪐 The universe is fundamentally discrete (quantum mechanics), but at a high level it's approximately continuous (classical mechanics) 🤖 🧠 Cognitive science 🧠 🤖 The mind is fundamentally continuous (the brain), but at a high level it's approximately discrete (symbols)

Since the Odyssey is in the news, here's an anecdote: When I was 14 or so, I noticed a book on our kitchen table with the title ODYSSEY. But a couple things about the book were odd... 1/3

Next was an excellent talk by @rtommccoy.bsky.social on using training data statistics to examine how many LLM generations are exact replicas of the training data at the Simons Institute www.youtube.com/watch?v=3c9T... (4/7

I like the NLP reference in "Walking in a Winter Wonderland", where they say "In the meadow, we can build a snowman / Then pretend that he is parsin' [the] Brown [corpus]"

Transformer LMs get pretty far by acting like ngram models, so why do they learn syntax? A new paper by sunnytqin.bsky.social, me, and @dmelis.bsky.social illuminates grammar learning in a whirlwind tour of generalization, grokking, training dynamics, memorization, and random variation. #mlsky #nlp

This is an ice-cold take, but it's amazing to live in an era where a 6-year-old model (BERT) is considered so ancient that the new version gets called "ModernBERT"! (I'm excited to see new work on BERT-style models; it's been too long!)

Or perhaps the problem is we keep benchmarking these tools against human tasks rather than measuring them based on what they are designed to do. The Embers of Autoregression continue to singe. www.pnas.org/doi/10.1073/...

🚨 Thrilled to share that Compositional Generalization Across Distributional Shifts with Sparse Tree Operations received a spotlight award at #NeurIPS2024! 🌟 I'll present a poster on Tuesday and give an invited lightning talk at the System 2 Reasoning Workshop on Sunday. 🧵👇

I love words like "nevertheless", "inasmuch", "whithersoever", "notwithstanding" Just throwing caution to the winds and cramming together as many adverbs & prepositions as we can

Excited to be visiting the Simons Institute tomorrow for a debate with Sébastien Bubeck - billed as the Sparks/Embers debate! 🔥🤖🧠 Topic: Will scaling current LLMs be sufficient to resolve major open math conjectures?

(1/5) Very excited to announce the publication of Bayesian Models of Cognition: Reverse Engineering the Mind. More than a decade in the making, it's a big (600+ pages) beautiful book covering both the basics and recent work: mitpress.mit.edu/978026204941...

🤖🧠 I'll be considering applications for postdocs & PhD students to start at Yale in Fall 2025! If you are interested in the intersection of linguistics, cognitive science, and AI, I encourage you to apply! Postdoc link: rtmccoy.com/prospective_... PhD link: rtmccoy.com/prospective_...

"Worldpay" - anagram of "Wordplay"!