love the construction of this dataset, it feels very much of a piece with other work like the reversal curse. LLMs clearly don't "learn" or "know" in the ways we do, whether relation reversal or simple compositionality, and I love how clearly this work demonstrates that.

My name is Ozymandias, King of Kings! Look upon my works, which achieve state-of-the art performance across a diverse suite of benchmarks

✨I’m on the academic job market ✨ I’m a PhD candidate at @hcii.cmu.edu studying tech, labor, and resistance 👩🏻‍💻💪🏽💥 I research how workers and communities contest harmful sociotechnical systems and shape alternative futures through everyday resistance and collective action More info: cella.io

How to not do computational humanities: (1) Lay out a question/hypothesis about a complex, cultural domain. (2) Compute numbers that were inspired by (1) but without sufficiently formalizing (1) so as to meaningfully link to it. (3) Interpret numbers to mean whatever your prior vibes were.

So excited to be TAing this course! So much of the knowledge you have as a PhD student is expected to be gained by osmosis, and it leads to some odd holes and gaps. This course should fix most of that problem!

The military keynesianism of our day (though it's of course possible that military keynesianism is the military keynesianism of our day, and this is just _another_ one)

Coming soon (6pm!) to the #ACL poster session: how do experts work with collections of documents, and do LLMs do those things? tl;dr: only sometimes! While we have good tools for things like information extraction, the way that experts read documents goes deeper - come to our poster to learn more!

Gently, I would like to say: When people tell you that they would appreciate a feature that does something automatically, it's not responsive to that concern to explain that by going through several steps for every individual instance, they can get the same result in each instance.

OpenAI has effectively conned people into thinking that Chatbots & AI "Assistants" are The FEWTCHA of AI. Friends, they are most likely *not.* Neither are the big cloud-based Generative AI services. Small, purpose-fit, on-device models that make your existing activities easier/better? There you go.

When it comes to text prediction, where does one LM outperform another? If you've ever worked on LM evals, you know this question is a lot more complex than it seems. In our new #acl2025 paper, we developed a method to find fine-grained differences between LMs: 🧵1/9

I for one am grateful for the opportunity to meditate on the meaning of “scientific artifact” at 2:15am

Great points in the replies here re:civility being an unequally applied, generally awful standard. It's also a great example of how common formulations of "sentiment analysis" or "toxicity detection" in NLP that are non-contextual lead to systems that stop sounding good with even slight scrutiny.

AI as another pivot to video is a thought I've also had, and this article is such a great articulation. The way "AI" is framed, pitched, and deployed by big players reflects at best ignorance, and at worst a real contempt for the social nature and humanity of our jobs and online spaces

This talk was such a joy to do! If you'd like to read the paper, it's here: arxiv.org/abs/2411.17840. Thank you for having us, @patrickbriansmith.bsky.social!

If you're at NAACL this week (or just want to keep track), I have a feed for you: bsky.app/profile/did:... Currently pulling everyone that mentions NAACL, posts a link from the ACL Anthology, or has NAACL in their username. Happy conferencing!

Ever trusted a metric that works great on average, only for it to fail in your specific use case? In our #NAACL2025 paper (w/ @841io.bsky.social), we show why global evaluations are not enough and why context matters more than you think. 📄 aclanthology.org/2025.finding... #NLP #Evaluation (🧵1/9)

🚀 Excited to share a new interp+agents paper: 🐭🐱 MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools appearing at #NAACL2025 This was work done @msftresearch.bsky.social last summer with Jason Eisner, Justin Svegliato, Ben Van Durme, Yu Su, and Sam Thomson 1/🧵

This is absurdly great, but I haven't read a single news article about it. A fully open source, offline-first alternative to Notion that's a collab between the French and German governments because they want to host docs securely and on their own terms. THIS is what Europe should be doing.

one thing in AI is not new -- people taking one small part of a job, mischaracterizing it, ignoring all the other stuff, and then assume the AI can do the whole job on its own

Have these people met … society? Read a book? Listened to music? Regurgitating esoteric facts isn’t intelligence. This is more like humanity’s last stand at jeopardy www.nytimes.com/2025/01/23/t...

[[ Now that it's not 1st anymore and more people might pay attention, sharing this again. ]] Hello *friends* doing ai + culture work, send across your burning questions about research, career, field, etc. It is anonymous :)

When I started on ARL project that funds my PhD, the thing we were supposed to build was a "MaterialsGPT". What is a MaterialsGPT? Where does that idea come from? I got to spend a lot of time thinking about that second question with @davidthewid.bsky.social and Lucy Suchman (!) working on this:

As someone whose PhD would not have been possible without military funding, this paper was a privilege to write. (Even outside of continuing to work with @davidthewid.bsky.social, and getting to work with Lucy Suchman (!))

Fully dislocated my shoulder going down some icy steps, so there will be no winter fishing for me this year :/ now gazing longingly at pictures of the last time I was out

Really interesting story! newbedfordlight.org/what-happene...

Just saw Wicked, and finally understood all of Glinda's lyrics because they were being sung by...*checks notes*...Ariana Grande???

Proposals like llms.txt (llmstxt.org) are such a fascinating rhyme of the semantic web. The standard for machine readability is so much more flexible right now, but I'd argue that we only really got here because we abandoned the original promise of upfront, user-defined machine readability!

This really changes the game for when my loved ones and I do our weekly comparison. Awful to know we've been overcompensating for frequency and length 😓

Looking for all your LTI friends on Bluesky? The LTI Starter Pack is here to help! go.bsky.app/NhTwCVb

Today is the day!! Find me at 2 PM in the Jasmine Hall (the one on the floor near food).

I've been missing paper recommendations as a regular part of my feed, so I threw together a feed that pulls every skeet (minus link repost bots) with a link to arXiv, the ACL Anthology, NeurIPS proceedings, or OpenReview: bsky.app/profile/did:... It's noisy, but maybe that's a feature?

Hell yeah 😎

I'm keeping track of people at the CMU Language Technologies Institute here: go.bsky.app/NhTwCVb. Follow along!

This was an excellent series from an excellent podcast, highly recommend!

VLADIMIR: Well? Shall we log off? ESTRAGON: Yes, let's log off. They do not log off.

me watching my advisor adjust vspace in the overleaf

"His specialty was alfalfa, and he made a good thing out of not growing any. The government paid him well for every bushel of alfalfa he did not grow. The more alfalfa he did not grow, the more money the government gave him"

Well it's called a "personal research plan" in my department

The problem isn't that AI can do your job (it can't). The problem is management thinks AI can do your job.

Me: AI assistant, decline meetings AI: meetingae, meetingās, meetingārum, meetingīs

I've got an amazing joke about Polyphemus, but nobody to tell it to :(

an earthquake three days before a total eclipse of the sun would absolutely have taken out a whole ass dynasty back in the day

Computer scientist Joseph Weizenbaum (1975) on why there was no "computer revolution" because computers are politically conservative and prevented social change that could have otherwise occurred...

I need to write the whole piece about this, but the thing to understand is that the NYT is now just Facebook. The platform dictates narrative to normies, is totally gamed by the right, and is still so ubiquitous as to be unavoidable even by those who see how broken it is. It's just Facebook.

We all know that “recently large language models have”, “large language models are”, and “large language models can.” But *why* LLMs? How did we get here? (where is “here”?) What forces are shaping NLP, and how recent are they, actually? To appear at EMNLP 2023: arxiv.org/abs/2310.07715