The project that started my whitespace obsession... #EMNLP2025 While we've all been worrying about tokenizers, lurking in the background has been the preprocessing *before* tokenization. Poems break standard HTML-to-text linearization systems, and we find that multimodal models aren't a solution.

𝙒𝙚'𝙧𝙚 𝙝𝙞𝙧𝙞𝙣𝙜 𝙣𝙚𝙬 𝙛𝙖𝙘𝙪𝙡𝙩𝙮 𝙢𝙚𝙢𝙗𝙚𝙧𝙨! KSoC: utah.peopleadmin.com/postings/190... (AI broadly) Education + AI: - utah.peopleadmin.com/postings/189... - utah.peopleadmin.com/postings/190... Computer Vision: - utah.peopleadmin.com/postings/183...

Which, whose, and how much knowledge do LLMs represent? I'm excited to share our preprint answering these questions: "Epistemic Diversity and Knowledge Collapse in Large Language Models" 📄Paper: arxiv.org/pdf/2510.04226 💻Code: github.com/dwright37/ll... 1/10

Go check Alex's poster today (Wed) in Suzhou! #EMNLP2025 I'm still so proud of our work (led by @lasha.bsky.social) on CondaQA, so we had to ask what would happen if we tried to create high-quality reasoning-over-text benchmarks now that LLMs are available. Turns out, we'd make an easier benchmark!

I'll be in Suzhou 🇨🇳 at #EMNLP this week presenting "What has been Lost with Synthetic Evaluation?" done with @anamarasovic.bsky.social & @lasha.bsky.social! 🎉 📍Findings Session 1 - Hall C 📅 Wed, November 5, 13:00 - 14:00 arxiv.org/abs/2505.22830

AI always calling your ideas “fantastic” can feel inauthentic, but what are sycophancy’s deeper harms? We find that in the common use case of seeking AI advice on interpersonal situations—specifically conflicts—sycophancy makes people feel more right & less willing to apologize.

Interested in language models, brains, and concepts? Check out our COLM 2025 🔦 Spotlight paper! (And if you’re at COLM, come hear about it on Tuesday – sessions Spotlight 2 & Poster 2)!

LLMs are trained to mimic a “true” distribution—their reducing cross-entropy then confirms they get closer to this target while training. Do similar models approach this target distribution in similar ways, though? 🤔 Not really! Our new paper studies this, finding 4-convergence phases in training 🧵

It is PhD application season again 🍂 For those looking to do a PhD in AI, these are some useful resources 🤖: 1. Examples of statements of purpose (SOPs) for computer science PhD programs: cs-sop.org [1/4]

Happy to see this work accepted to #EMNLP2025! 🎉🎉🎉

I am recruiting emergency reviewers for *SEM 2025 (The 14th Joint Conference on Lexical and Computational Semantics). Please DM me if you might be able to contribute a review within the next few days 🙏

We’re thrilled to congratulate Dr. Abhilasha Ravichander (@lasha.bsky.social) and her team for receiving the Outstanding Paper Award at #acl2025 for their work titled "HALoGEN: Fantastic LLM Hallucinations and Where to Find Them"! 🏆✨ #ACL #LLMs #Hallucination #WiAIR #WomenInAI

Status Update: I'm in the middle of my move from Denmark to Colorado! If I seem to be missing for the 1-2 weeks, that is the main reason. Picture me lost amidst suitcases, boxes, moving pods, and far too many books. Copenhagen friends, I'm here for a couple more days! Please stop by P1 to say bye 🥺

🎙️ New Women in AI Research #WiAIR episode coming Aug 6! We talk to @lasha.bsky.social about LLM Hallucination, her award-winning HALoGEN benchmark, and how we can better evaluate hallucinations in language models. 👇 What’s inside: 1/

Director at Max Planck - a unique position! The Open Call for Expressions of Interest in Max Planck Directorships is open now and can be submitted by the 31st of October 2025. ➡️ mpg.de/directors - Please share the Open Call among potential candidates.

Super super thrilled that HALoGEN, our study of LLM hallucinations and their potential origins in training data, received an ✨Outstanding Paper Award✨ at ACL! Joint work w/i Shrusti Ghela*, David Wadden, and Yejin Choi bsky.app/profile/lash...

Join Abhilasha's lab, she is an awesome researcher and mentor! I can attest, being her collaborator was great fun 🤩

People applying for NLP PhDs, work with Abhilasha -- she is awesome!!

📣 Life update: Thrilled to announce that I’ll be starting as faculty at the Max Planck Institute for Software Systems this Fall! I’ll be recruiting PhD students in the upcoming cycle, as well as research interns throughout the year: lasharavichander.github.io/contact.html

I'm sadly not at #IC2S2 😭, but I will be at #ACL2025 in Vienna ☕️ next week!! Please spread the word that I'm recruiting prospective PhD students: lucy3.notion.site/for-prospect...

💡Beyond math/code, instruction following with verifiable constraints is suitable to be learned with RLVR. But the set of constraints and verifier functions is limited and most models overfit on IFEval. We introduce IFBench to measure model generalization to unseen constraints.

𝐖𝐡𝐚𝐭 𝐇𝐚𝐬 𝐁𝐞𝐞𝐧 𝐋𝐨𝐬𝐭 𝐖𝐢𝐭𝐡 𝐒𝐲𝐧𝐭𝐡𝐞𝐭𝐢𝐜 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧? (arxiv.org/abs/2505.22830) I'm happy to announce that the preprint release of my first project is online! Developed with the amazing support of @lasha.bsky.social & @anamarasovic.bsky.social

🚨 Preprint alert 🚨 𝐂𝐚𝐧 𝐂𝐨𝐦𝐦𝐮𝐧𝐢𝐭𝐲 𝐍𝐨𝐭𝐞𝐬 𝐑𝐞𝐩𝐥𝐚𝐜𝐞 𝐏𝐫𝐨𝐟𝐞𝐬𝐬𝐢𝐨𝐧𝐚𝐥 𝐅𝐚𝐜𝐭-𝐂𝐡𝐞𝐜𝐤𝐞𝐫𝐬? (arxiv.org/abs/2502.14132) Fact-checking agencies have come under intense scrutiny in recent months regarding their role in combating misinformation on social media.

*SEM 2025: Direct submissions deadline is now EXTENDED to June 13, 2025! *SEM will be co-located with EMNLP in Suzhou, China. It will be a hybrid event, virtual attendance possible (more info on this will be posted later). Full CFP: starsem2025.github.io/cfp #NLP #NLProc

HALoGEN will appear at #ACL2025, see you in Vienna!

This is an awesome awesome-llm-unlearning resource, maintained by Chris Liu : github.com/chrisliu298/...

We are launching HALoGEN💡, a way to systematically study *when* and *why* LLMs still hallucinate. New work w/ Shrusti Ghela*, David Wadden, and Yejin Choi 💫 📝 Paper: arxiv.org/abs/2501.08292 🚀 Code/Data: github.com/AbhilashaRav... 🌐 Website: halogen-hallucinations.github.io 🧵 [1/n]

Come catch our talk today at 5 PM in San Miguel!

I'm late to the NAACL party, but I've just arrived in Albuquerque! I'll talk about measuring faithfulness of verbalized reasoning on *Sunday* at Repl4NLP at *9:45*.

Sadly I'm not at #NAACL2025 but @lasha.bsky.social is there to present this paper, which I helped with a bit and really like! Find her if you'd like to chat about memorization!

If you're at NAACL this week (or just want to keep track), I have a feed for you: bsky.app/profile/did:... Currently pulling everyone that mentions NAACL, posts a link from the ACL Anthology, or has NAACL in their username. Happy conferencing!

✈️ I'm in New Mexico for #NAACL2025! Would love to meet and chat about training data, factuality, transparency, robustness, doing a PhD in AI🤖, or anything else. I love to make friends, please say hi if you see me! 🌵🌞 And check out our work at NAACL👇

Stoked that HALoGEN (non-archival version) won best paper award at the TrustNLP workshop @ #NAACL2025! Our work explores LLM hallucinations and potential roots in training data. Excited to discuss more --- come find us!

I'll be at #NAACL2025: 🖇️To present my paper "Superlatives in Context", showing how the interpretation of superlatives is very context dependent and often implicit, and how LLMs handle such semantic underspecification 🖇️And we will present RewardBench on Friday Reach out if you want to chat!

Announcing our first workshop keynote! "Processing Trans Languaging" by Vagrant Gautam @dippedrusk.com at 13:30 - 14:30 MDT on May 4, 2025 Read on for a description of the talk! 🧵

💡 New ICLR paper! 💡 "On Linear Representations and Pretraining Data Frequency in Language Models": We provide an explanation for when & why linear representations form in large (or small) language models. Led by @jackmerullo.bsky.social, w/ @nlpnoah.bsky.social & @sarah-nlp.bsky.social

I'm in Singapore for @iclr-conf.bsky.social ! Come check out our spotlight paper on the environmental impact of training OLMo (link in next tweet) during the Saturday morning poster session from 10-12:30 -- happy to chat about this or anything else! DMs should be open, email works too

After a short era in which people questioned the value of academia in ML, its value is more obvious than ever. Big labs stopped publishing the minute commercial incentives showed up and are relentlessly focused on a singular vision of scaling. Academia is a meaningful complement, bringing... 1/2

Me: someday my research will be mentioned in the opening paragraph of a New York Times article! Also me: hmmm should have been more specific. www.nytimes.com/2025/04/22/s...

One of my PhD students got their visa revoked. I know of other cases amongst my AI colleagues. This is not what investing in US leadership in AI looks like. www.aljazeera.com/news/2025/4/...

Can confirm that my NSF grant "How False Beliefs Form & How to Correct Them" was cancelled today because it is "not in alignment with current NSF priorities" Shocking that understanding how people are misled by false information is now a forbidden topic. Our work will continue but at a smaller scale

🚨Report your NSF grant terminations! 🚨 We are starting to collect information on NSF grant terminations to create a shared resource as we have for NIH. The more information we collect, the more we can organize, advocate, and fight back! Please share widely! airtable.com/appGKlSVeXni...

🔭 Science relies on shared artifacts collected for the common good. 🛰 So we asked: what's missing in open language modeling? 🪐 DataDecide 🌌 charts the cosmos of pretraining—across scales and corpora—at a resolution beyond any public suite of models that has come before.

idk who needs to see this but here is the photo that appears on the american pika's wikipedia page

Today we're unveiling OLMoTrace, a tool that enables everyone to understand the outputs of LLMs by connecting to their training data. We do this on unprecedented scale and in real time: finding matching text between model outputs and 4 trillion training tokens within seconds. ✨

For years it’s been an open question — how much is a language model learning and synthesizing information, and how much is it just memorizing and reciting? Introducing OLMoTrace, a new feature in the Ai2 Playground that begins to shed some light. 🔦

We have also added #Maverick which is arguably better but (similarly to other models) struggles when the evidence is not easy to retrieve (ie requires more reasoning than simple retrieval -- hard set). More here: novelchallenge.github.io

Llama 4's massive context window is impressive! However, the best Llama model for long-context understanding over books is still Llama 3.1 405B. Llama 4 Scout is especially bad at our NoCha benchmark, performing below random chance.