Thinking about my favorite amp today 😔❤️

🚀 SynthTextEval, our open-source toolkit for generating and evaluating synthetic text data for high-stakes domains, will be featured at EMNLP 2025 as a system demonstration! GitHub: github.com/kr-ramesh/sy... Paper 📝: aclanthology.org/2025.emnlp-d... #EMNLP2025 #EMNLP #SyntheticData

In honor of some new people coming from AI twitter, I finally updated my post to recommend For You over Discover.

A company that believed it was in the verge of AGI or ASI wouldn’t capitulate to the government because it wouldn’t care about government contracts. They would soon BE the economy and the government would soon be capitulating to them.

Keynote spotlight #4: the second day of COLM will close with @ghadfield.bsky.social from JHU talking about human society alignment, and lessons for AI alignment

Congratulations to Alane Suhr '22, a #CornellTech Ph.D. #alumni advised by associate professor Yoav Artzi, for receiving the prestigious 2022 @aaai.org / @acmsigai.bsky.social Doctoral Dissertation Award! Read more about the award here: aaai.org/about-aaai/a... @yoavartzi.com

Time for the world to install a gigawatt of solar power capacity 2004: A year 2010: ~ a month 2015: ~ a week Now: A day ourworldindata.org/data-insight... 🧪

🚨 Urban Stats 28.0.0 🚨 The mapper is now completely redesigned by me and @spudwaffle.bsky.social, allowing for much prettier looking maps and way more customization alongside significantly more options for geographies! See below for some of the examples of the maps you can create!

When reading AI reasoning text (aka CoT), we (humans) form a narrative about the underlying computation process, which we take as a transparent explanation of model behavior. But what if our narratives are wrong? We measure that and find it usually is. Now on arXiv: arxiv.org/abs/2508.16599

Paper: arxiv.org/pdf/2505.22037 🔗 Project page: aka.ms/jailbreak-d... 📊 Dataset: huggingface.co/datasets/ja...

So, what's the future of AI safety benchmarks? Jack's solution is "renewable benchmarks" that allows us to refresh and expand benchmarks with a single click!! x.com/jackjingyuz...

In our forthcoming paper, John Hummel and I ask what it would mean for a neural computing architecture such as a brain to implement a symbol system, and the related question of what makes it difficult for them to do so, with an eye toward the differences between humans, animals, and ANNs.

This paper is making the rounds: arxiv.org/abs/2506.21734 A tiny (27M) brain-inspired model trained just on 1000 samples outperforming o3-mini-high on reasoning tasks. #MLSky 🧠🤖

Interested in large-scale GPU optimization? Interested in how modern neural networks are being deployed to solve classical optimization problems? Writing a paper on these topics? Submit to the ScaleOPT workshop at NeurIPS! www.cvxgrp.org/scaleopt/#su...

I'm recruiting MLEs @ #ACL2025! Reach out if you know folks interested in legal NLP, structured prediction, and full-time at a startup environment in NYC I'll also always chat about: • population-level inference on corpora • broad-coverage semantics • which café has the best Sachertorte in Vienna

My students and I are presenting three papers on Monday at #ACL2025 and this thread will recap them (including their videos).

“Wikipedia is this economic anomaly. In many ways, it’s sort of magical that people will just volunteer without explicit economic incentives to create artifacts that are meant to share knowledge with everyone in the world”

Taking off for Vienna #ACL2025! 🇦🇹 Excited to talk with people about transparent reasoning, multimodality, and fact verification. Stop by our multimodal RAG workshop on Friday 🔥🔥🔥 Please reach out if you want to grab coffee!

The #ACL2025 #ACL2025NLP feed is up and running! It matches both hashtags and any posts from or mentions of @aclmeeting.bsky.social Pin it to your home 📌 and enjoy! bsky.app/profile/did:...

Juxtastat DAU update! Crazy how we've been >1000 every day for over a year now! Thank you all for all your support, and make sure to keep spreading the word!

🥳 🎉 ❤️ The ACL 2025 Proceedings are live on the ACL Anthology 🥰 ! We’re thrilled to pre-celebrate the incredible research 📚 ✨ that will be presented starting Monday next week in Vienna 🇦🇹 ! Start exploring 👉 aclanthology.org/events/acl-2... #NLProc #ACL2025NLP #ACLAnthology

This New Yorker piece is the most hopeful I've felt about the world in a long time. I had no idea solar was booming like this. And if you live in the same world as me, dominated by oil & gas guys maintaining that solar and wind are inefficient gimmicks, you might not've known some of this either.

🔈When LLMs solve tasks with a mid-to-low resource input or target language, their output quality is poor. We know that. But can we put our finger on what breaks inside the LLM? We introduce the 💥 translation barrier hypothesis 💥 for failed multilingual generation with LLMs. arxiv.org/abs/2506.22724

I wrote something up for AI people who want to get into bluesky and either couldn't assemble an exciting feed or gave up doomscrolling when their Following feed switched to talking politics 24/7.

Explore Wikipedia through a data map. Pages are grouped by semantic similarity, for topic clusters. Hover to see details, zoom to explore more fine-grained topics, click to go to a page. Search by page name to find interesting starting points for exploration. lmcinnes.github.io/datamapplot_...

William Walden, Kathryn Ricci, Miriam Wanner, Zhengping Jiang, Chandler May, Rongkun Zhou, Benjamin Van Durme How Grounded is Wikipedia? A Study on Structured Evidential Support https://arxiv.org/abs/2506.12637

What happens when an LLM is asked to use information that contradicts its knowledge? We explore knowledge conflict in a new preprint📑 TLDR: Performance drops, and this could affect the overall performance of LLMs in model-based evaluation.📑🧵⬇️ 1/8 #NLProc #LLM #AIResearch

Learn about the groundbreaking work being presented by JHU researchers at @cvprconference.bsky.social’s #CVPR2025! Check out the full list here www.cs.jhu.edu/news/johns-h... or browse through the thread below! 🧵 (1/14)

We know that speech LID systems flunk on accented speech. But why? And what can we do about it? 🤔 Our work arxiv.org/abs/2506.00628 (Interspeech '25) finds that *accent-language confusion* is an important culprit, ties it to the length of feature that the model relies on, and proposes a fix.

Excited to announce that I'm working on a project at AWS in New York this summer! Reach out if you're in the area and want to grab coffee 😀

@npr.org for every minute spent talking to a non-autistic person about autistic people's needs, you should be giving at least 2x that amount to actually autistic people— & not just in written stories, but on-air. Apply said rubric to any other group under attack right now for their lived realities.

We had a ton of fun last summer hacking away on this problem at the SCALE 2024 summer workshop. This summer, we're bringing it to Vienna as an ACL 2025 shared task!

🚨 IT'S HERE! 🚨 The Eval Leaderboard is now LIVE! 🏆💻 Our video retrieval collection stumps most pre-trained models. See if you can build a better system! eval.ai/web/challeng...

First draft online version of The RLHF Book is DONE. Recently I've been creating the advanced discussion chapters on everything from Constitutional AI to evaluation and character training, but I also sneak in consistent improvements to the RL specific chapter. rlhfbook.com

THIS IS ONE NEURON! Jaw dropping. It distributes a particular neurotransmitter (norepinephrine) across the mouse brain; it’s a locus coeruleus neuron. @jeremiahycohen.bsky.social and colleagues at the @alleninstitute.bsky.social are using new biotech to see things never seen before.

So, we wrote a neural net library entirely in LaTeX...

🚨 New preprint on transparent, tree-adaptive grounded reasoning! We introduce Bonsai, a versatile reasoning system that generates interpretable, grounded, and uncertainty-aware reasoning traces while enabling state-of-the-art performance on text and video benchmarks.

🚨 Deadline extension! 🚨 The MAGMaR paper submission date has been extended to May 1, 2025 (archival and non-archival). Show off your multimodal + RAG projects!

Really enjoyed diving deep into this paper today: arxiv.org/abs/2406.16860 It's so systematic and a treasure trove full of insights

ByteDance's Recitation over Reasoning The cutting-edge LLMs unanimously exhibits extremely severe recitation behavior; by changing one phrase in the condition, top models such as OpenAI-o1 and DeepSeek-R1 can suffer 60% performance loss on elementary school-level arithmetic and reasoning problems.

Introducing the DeepSeek-R1 Thoughtology -- the most comprehensive study of R1 reasoning chains/thoughts ✨. Probably everything you need to know about R1 thoughts. If we missed something, please let us know.

🎉 Our Actionable Interpretability workshop has been accepted to #ICML2025! 🎉 > Follow @actinterp.bsky.social > Website actionable-interpretability.github.io @talhaklay.bsky.social @anja.re @mariusmosbach.bsky.social @sarah-nlp.bsky.social @iftenney.bsky.social Paper submission deadline: May 9th!

Want to know what training data has been memorized by models like GPT-4? We propose information-guided probes, a method to uncover memorization evidence in *completely black-box* models, without requiring access to 🙅‍♀️ Model weights 🙅‍♀️ Training data 🙅‍♀️ Token probabilities 🧵 (1/5)

Put together some takes and a reading list on all the GRPO discussions happening recently (DAPO and Dr. GRPO). Essentially, this is me yapping through a recap of smaller details on how GRPO is implemented, what Dr. GRPO changes, why, DAPO, connections to PPO, aggregating batches.. buff.ly/S27ORh4

🚀 "Multilingual" LLMs are really just clusters of monolingual ones! They might know a Brazilian 🇧🇷 brewery—but only in Portuguese With ECLEKTIC, you can now test this. The challenge? Making them truly multilingual. alphaxiv.org/pdf/2502.21228 📈🤖 🧵⬇️ #LLM #ai #genAI #nlp

Want to check out the source for the "AlexNet" paper? Google has made the code from Krizhevsky, Sutskever and Hinton's seminal "ImageNet Classification with Deep Convolutional Neural Networks" paper open source, in partnership with the Computer History Museum. computerhistory.org/press-releas...

A bit of a mess around the conflict of COLM with the ARR (and to lesser degree ICML) reviews release. We feel this is creating a lot of pressure and uncertainty. So, we are pushing our deadlines: Abstracts due March 22 AoE (+48hr) Full papers due March 28 AoE (+24hr) Plz RT 🙏

You can't just be right, you have to know you're right. Good advice for LLMs, according to new Johns Hopkins research. Sometimes no answer is better than a wrong one - life or death choices in medicine, for example, or big financial decisions. 🧵

I've recently put together a "Fairness FAQ": tinyurl.com/fairness-faq. If you work in non-fairness ML and you've heard about fairness, perhaps you've wondered things like what the best definitions of fairness are, and whether we can train algorithms that optimize for it.