Good news everyone! I’ll be presenting the paper I did with Marco and Iuri "Prediction Hubs are Context-Informed Frequent tokens in LLMs" at the ELLIS UnConference on December 2nd in Copenhagen. arxiv.org/abs/2502.10201

Do you use a pronoun more often when the entity you’re talking about is more predictable? Previous work offers diverging answers so we conducted a meta-analysis, combining data from 20 studies across 8 different languages. Now out in Language: muse.jhu.edu/article/969615

First paper is out! Had so much fun presenting it in Marseille last July 🇨🇵 We explore how transformers handle compositionality by exploring the representations of the idiomatic and literal meaning of the same noun phrase (e.g. "silver spoon"). aclanthology.org/2025.jeptaln...

New paper! 🚨 I argue that LLMs represent a synthesis between distributed and symbolic approaches to language, because, when exposed to language, they develop highly symbolic representations and processing mechanisms in addition to distributed ones. arxiv.org/abs/2502.11856

Our paper "Prediction Hubs are Context-Informed Frequent Tokens in LLMs" has been accepted at ACL 2025! Main points: 1. Hubness is not a problem when language models do next-token prediction. 2. Nuisance hubness can appear when other comparisons are made.

The @interspeech.bsky.social early registration deadline is coming up in a few days! Want to learn how to analyze the inner workings of speech processing models? 🔍 Check out the programme for our tutorial: interpretingdl.github.io/speech-inter... & sign up through the conference registration form!

Last day to sign up for the COLT Symposium! Register: tinyurl.com/colt-register 📢 𝗟𝗼𝗰𝗮𝘁𝗶𝗼𝗻 𝗰𝗵𝗮𝗻𝗴𝗲📢 June 2nd, 14:30 - 19:00 UPF Campus de la Ciutadella Room 40.101 maps.app.goo.gl/1216LJRsWmTE...

⭐ Registration open til May 27th! ⭐ Website: www.upf.edu/web/colt/sym... June 2nd, UPF 𝗦𝗽𝗲𝗮𝗸𝗲𝗿 𝗹𝗶𝗻𝗲𝘂𝗽: Arianna Bisazza (language acquisition with NNs) Naomi Saphra (emergence in LLM training dynamics) Jean-Rémi King (TBD) Louise McNally (pitfalls of contextual/formal accounts of semantics)

🧵 Excited to share our paper "Unique Hard Attention: A Tale of Two Sides" with Selim, Jiaoda, and Ryan, where we show that the way transformers break ties in attention scores has profound implications on their expressivity! And it got accepted to ACL! :) The paper: arxiv.org/abs/2503.14615

Announcing the COLT Symposium on June 2nd! 𝗘𝗺𝗲𝗿𝗴𝗲𝗻𝘁 𝗳𝗲𝗮𝘁𝘂𝗿𝗲𝘀 𝗼𝗳 𝗹𝗮𝗻𝗴𝘂𝗮𝗴𝗲 𝗶𝗻 𝗺𝗶𝗻𝗱𝘀 𝗮𝗻𝗱 𝗺𝗮𝗰𝗵𝗶𝗻𝗲𝘀 What properties of language are emerging from work in experimental and theoretical linguistics, neuroscience & LLM interpretability? Info: tinyurl.com/colt-site Register: tinyurl.com/colt-register 🧵1/3

🌍📣🥳 I could not be more excited for this to be out! With a fully automated pipeline based on Universal Dependencies, 43 non-Indoeuropean languages, and the best LLMs only scoring 90.2%, I hope this will be a challenging and interesting benchmark for multilingual NLP. Go test your language models!

NYU canceled an invited talk by the former president of Doctors Without Borders, out of fear her talk would be accused by the government of being both anti-Trump and antisemitic: ici.radio-canada.ca/nouvelle/215...

new pre-print: LLMs as a synthesis between symbolic and continuous approaches to language arxiv.org/abs/2502.11856

The project I did with Marco Baroni and Iuri Macocco while I was in Barcelona is now on Arxiv: arxiv.org/abs/2502.10201 🎉 TLDR below 👇

The DOGE firings have nothing to do with “efficiency” or “cutting waste.” They’re a direct push to weaken federal agencies perceived as liberal. This was evident from the start, and now the data confirms it: targeted agencies overwhelmingly those seen as more left-leaning. 🧵⬇️

🚨BREAKING. From a program officer at the National Science Foundation, a list of keywords that can cause a grant to be pulled. I will be sharing screenshots of these keywords along with a decision tree. Please share widely. This is a crisis for academic freedom & science.

Here's our work accepted to #ICLR2025! We look at how intrinsic dimension evolves over LLM layers, spotting a universal high-dimensional phase. This ID peak is where: - linguistic features are built - different LLMs are most similar, with implications for task transfer 🧵 1/6

I think some people hear “grants” and think that without them, scientists and government workers just have less stuff to play with at work. But grants fund salaries for students, academics, researchers, and people who work in all areas of public service. “Pausing” grants means people don’t eat.

🔊New EMNLP paper from Eleonora Gualdoni & @gboleda.bsky.social ! Why do objects have many names? Human lexicons contain different words that speakers can use to refer to the same object, e.g., purple or magenta for the same color. We investigate using tools from efficient coding...🧵 1/3

⚡Postdoc opportunity w/ COLT Beatriu de Pinós contract, 3 yrs, competitive call by Catalan government. Apply with a PI (Marco Gemma or Thomas) Reqs: min 2y postdoc experience outside Spain, not having lived in Spain for >12 months in the last 3y. Application ~December-February (exact dates TBD)

Hello🌍! We're a computational linguistics group in Barcelona headed by Gemma Boleda, Marco Baroni & Thomas Brochhagen We do psycholinguistics, cogsci, language evolution & NLP, with diverse backgrounds in philosophy, formal linguistics, CS & physics Get in touch for postdoc, PhD & MS openings!

My lab has been working on comparing neural representations for the past few years - methods like RSA, CKA, CCA, Procrustes distance We are often asked: What do these things tell us about the system's function? How do they relate to decoding? Our new paper has some answers arxiv.org/abs/2411.08197