Grok fact-checks our paper on Grok fact-checking - and it approves!

🎭 How do LLMs (mis)represent culture? 🧮 How often? 🧠 Misrepresentations = missing knowledge? spoiler: NO! At #CHI2026 we are bringing ✨TALES✨ a participatory evaluation of cultural (mis)reps & knowledge in multilingual LLM-stories for India 📜 arxiv.org/abs/2511.21322 1/10

this is amazing! made quick NYC & boston posters

Potato is a great platform for researchers! Highly recommend (plus a great development team behind it)

Microsoft Research NYC is hiring a researcher in the space of AI and society!

What should academics be doing right now? I have been writing up some thoughts on what the research says about effective action, and what universities specifically can do. davidbau.github.io/poetsandnurs... It's on GitHub. Suggestions and pull requests welcome. github.com/davidbau/poe...

I'll be presenting this work on January 25th (Hall 2, poster 41) at #AAAI2026 in Singapore! Please stop by and reach out if you'd like to chat 😁

🎉 Excited to share our new paper which was accepted to #AAAI2026! As LLMs become increasingly used as sources of factual knowledge, we ask: Do they perform equitably across users of different backgrounds? 🧵⬇️ 1/6

Most LLM evals use API calls or offline inference, testing models in a memory-less silo. Our new Patterns paper shows this misses how LLMs actually behave in real user interfaces, where personalization and interaction history shape responses: arxiv.org/abs/2509.19364

Elinor Poole-Dayan, Jiayi Wu, Taylor Sorensen, Jiaxin Pei, Michiel A. Bakker: Benchmarking Overton Pluralism in LLMs https://arxiv.org/abs/2512.01351 https://arxiv.org/pdf/2512.01351 https://arxiv.org/html/2512.01351

Thoughtful (as always) blog post from Nicholas Carlini. "Are large language models worth it?" A nice read giving his perspective on risks of ML models. Post: nicholas.carlini.com/writing/2025... For people who prefer, this is the video of the talk from @colmweb.org www.youtube.com/watch?v=PngH...

Extremely thrilled to talk about our new paper: "Who Evaluates AI’s Social Impacts? Mapping Coverage And Gaps In First And Third Party Evaluations". This is the first big project output from the @eval-eval.bsky.social coalition! Thread below:

Congratulations @sivareddyg.bsky.social ! 🥳 Incredibly well deserved!!

We're excited to announce that the website and registration for IC2S2 2026 (July 28-31) will launch in early December! The Vermont Complex Systems Institute @vcsi.bsky.social at the University of Vermont will be hosting IC2S2 in 2026: youtube.com/watch?v=p412S4GnPkc&feature=youtu.be

It's the season for PhD apps!! 🥧 🦃 ☃️ ❄️ Apply to Wisconsin CS to research - Societal impact of AI - NLP ←→ CSS and cultural analytics - Computational sociolinguistics - Human-AI interaction - Culturally competent and inclusive NLP with me! lucy3.github.io/prospective-...

It’s grad school application season, and I wanted to give some public advice. Caveats: -*-*-*-* 
> These are my opinions, based on my experiences, they are not secret tricks or guarantees 
> They are general guidelines, not meant to cover a host of idiosyncrasies and special cases

🧵Excited to present our work at #EMNLP2025 “Analyzing Dialectal Biases in LLMs for Knowledge and Reasoning Benchmarks”! Paper 📄 arxiv.org/abs/2510.00962 w/ Eileen Pan, Skyler Seto, @allisonkoe.bsky.social @maartjeterhoeve.bsky.social

it's crazy how typos are impossible to catch until *after* you submit a paper, after which they become glaringly noticeable

Which, whose, and how much knowledge do LLMs represent? I'm excited to share our preprint answering these questions: "Epistemic Diversity and Knowledge Collapse in Large Language Models" 📄Paper: arxiv.org/pdf/2510.04226 💻Code: github.com/dwright37/ll... 1/10

What makes dialogue 💬 constructive 🫂? We address this question in our #EMNLP2025 paper investigating how **responsivity** can characterize conversation quality ✨ Brandon Roy will be presenting our work (Oral) on Nov 7, Room A109 at 10:30. 🧵👇 ​​https://aclanthology.org/2025.emnlp-main.1798/

Great piece by @natolambert.bsky.social on the current state of human exhaustion in the AI world. Makes this important point:

👀 www.bbc.co.uk/mediacentre/...

This paper underscores how “misinformation” is not easily operationalized as single pieces of factually untrue content, but often takes shape through the motivated amplification of specific evidence (often true evidence) that serves to build and reinforce misleading frames.

Really proud bc I finished my first crochet sweater! It’s pretty cool wearing something you made by hand, every single stitch, mistakes and all

MIT rejects "compact" proposed by the Trump administration. MIT prez wrote: it "would restrict freedom of expression and our independence as an institution" and "is inconsistent with our core belief that scientific funding should be based on scientific merit alone." orgchart.mit.edu/letters/rega...

Hello #COLM2025! Excited to be kicking off the NLP4Democracy workshop this morning. We are in 520E (behind A/B/C) - check out our amazing program! sites.google.com/andrew.cmu.e...

💡We kicked off the SoLaR workshop at #COLM2025 with a great opinion talk by @michelleding.bsky.social & Jo Gasior Kavishe (joint work with @victorojewale.bsky.social and @geomblog.bsky.social ) on "Testing LLMs in a sandbox isn't responsible. Focusing on community use and needs is."

Hi #COLM2025! 🇨🇦 I will be presenting a talk on the importance of community-driven LLM evaluations based on an opinion abstract I wrote with Jo Kavishe, @victorojewale.bsky.social and @geomblog.bsky.social tomorrow at 9:30am in 524b for solar-colm.github.io Hope to see you there!

Important keynote by Nicholas Carlini with important calls to action for the research community! Ty for the helpful summary @mariaa.bsky.social

COLM word cloud. Yoav says it’s the year of reasoning, but evaluation is also huge.

I’m at #COLM2025! Would love to chat about anything related to pluralistic alignment, fairness evaluations, societal impacts of LLMs, etc 😊 You can also find me at the NLP4Democracy workshop giving a talk about my work analyzing democratic deliberation with LLMs Oct 10th!

Alright the evening sky, you’re utterly wondrous and fantastical, we get it, geez

Here’s a #COLM2025 feed! Pin it 📌 to follow along with the conference this week!

I will be at #COLM2025 this week, and would love to connect with folks interested in applications (and critiques) of language modeling in social science research! And join us for the NLP4Democracy workshop on Friday! sites.google.com/andrew.cmu.e... #NLP #NLProc #LLM #ComputationalSocialScience

Very excited that my paper with @katakeith.bsky.social is now out in @polanalysis.bsky.social. We investigate whether LLMs actually follow the instructions/definitions provided in codebooks, propose some diagnostics, and release a new evaluation dataset. www.cambridge.org/core/journal...

I wish students understood in most empirical AI research there’s a huge scientific advantage from being constitutionally excited by math vs intimidated, but very little additional gain from being actually “good” at math. Maybe they’d be less intimidated if they didn’t feel they had to be “good”.

🚨 New preprint! 🚨 Excited to share my work: An AI-Powered Framework for Analyzing Collective Idea Evolution in Deliberative Assemblies 🤖🗳️ I’ll be presenting this at @colmweb.org in the NLP4Democracy workshop! 🔗 arxiv.org/abs/2509.12577

"This suggests that LLM benchmark behavior may generalize less and less to non-benchmark settings, raising new concerns about ecological validity." super interesting

This paper yields the same conclusion as what @mustafasuleymanai.bsky.social recently posted on the danger of 'seemingly conscious AI'. mustafa-suleyman.ai/seemingly-co...

When reading papers, especially reviewing, I like to print and annotate as I read. I wish I could upload this to open review so authors can see smaller suggestions (typos, formatting errors) as well as smaller positive notes eg things I appreciated or found useful/interesting

First crochet project done! Super proud and excited for the next one

This new center strikes the right tone in approaching the AI alignment problem. alignmentalignment.ai

It never ceases to amaze me that some folks can just continue on with their evenings and not stop everything to gaze at the magnificence of the post-rain sunset (and ofc take a billion photos)!

This is a cool paper that suggests that AI agents can indeed be used for social science experiments, but that just using a chatbot isn't good enough, instead prompts developed based on social & game theory makes AI agent actions predictive of real human outcomes. benjaminmanning.io/files/optimi...

It’s cool that Massachusetts is getting a better flag but disappointing that we weren’t brave enough to select this entry from the competition

I had an intense summer so I’m recovering with a vacork. It’s the opposite of a workation: I go to the office but I don’t get anything done, just drink coffee and distract other people

I got the complaint in the horrific OpenAI self harm case the the NY Times reported today This is way way worse even than the NYT article makes it out to be OpenAI absolutely deserves to be run out of business

Adam Raine, 16, died from suicide in April after months on ChatGPT discussing plans to end his life. His parents have filed the first known case against OpenAI for wrongful death. Overwhelming at times to work on this story, but here it is. My latest on AI chatbots: www.nytimes.com/2025/08/26/t...

Our work on Detecting Suspense in Stories will appear at COLM'25 arxiv.org/abs/2508.15794 Do LLMs know when stories are suspenseful? We ran LLMs through a bunch of classical psychology suspense studies The answer: kinda, sorta? Honestly, better than I expected. But the results are nuanced... 1/