This is an excellent piece on the way that a culture of repurposed and thin measurement has really warped what it means to do science. The risk of AI here is not just intensifying the use of these metrics, but reifying those flawed standards into systems that make the culture harder to change.

As a PhD student in computer science, the kind of knowledge production that is being "innovated" away here is exactly what departments like mine could use a lot more of. This is a real loss for CMU.

"the constraint is the feature" is an excellent way to put this. General purpose tools are only general purpose until you use them for something!

What if technology didn’t feel so… hollow? Some friends and I just released a manifesto about a world where tech leaves us feeling nourished (along with an evolving list of theses about how we can build it) resonantcomputing.org

A NeurIPS feed! Pin it and follow along.

It's strange to be in a field wishing for the bubble around you to pop, but I think this post captures some of the why. AI as technology has so much cool and fun potential, and it's going to be so exciting to see it freed from being developed and pushed by the awful people at the top of tech.

This is so exciting!! As someone that works on scientific PDFs, the extent to which format constraints limit how we represent the complexity of the science we do is maddening. I hope other venues adopt things like this soon!

i don't agree with the attitude of some faculty that their job is to train people who can run the code, even if they don't understand the underlying physics i want a society of people who are trained to be curious about the code they're being asked to run and capable of understanding it, actually

Please respond to this survey if you have changed or have thought about changing your name in academic publishing! For any reason, whether it be transition, recognizability, marriage, privacy, immigration, cultural reasons, etc. Please RT for reach :)

love the construction of this dataset, it feels very much of a piece with other work like the reversal curse. LLMs clearly don't "learn" or "know" in the ways we do, whether relation reversal or simple compositionality, and I love how clearly this work demonstrates that.

My name is Ozymandias, King of Kings! Look upon my works, which achieve state-of-the art performance across a diverse suite of benchmarks

✨I’m on the academic job market ✨ I’m a PhD candidate at @hcii.cmu.edu studying tech, labor, and resistance 👩🏻‍💻💪🏽💥 I research how workers and communities contest harmful sociotechnical systems and shape alternative futures through everyday resistance and collective action More info: cella.io

How to not do computational humanities: (1) Lay out a question/hypothesis about a complex, cultural domain. (2) Compute numbers that were inspired by (1) but without sufficiently formalizing (1) so as to meaningfully link to it. (3) Interpret numbers to mean whatever your prior vibes were.

So excited to be TAing this course! So much of the knowledge you have as a PhD student is expected to be gained by osmosis, and it leads to some odd holes and gaps. This course should fix most of that problem!

The military keynesianism of our day (though it's of course possible that military keynesianism is the military keynesianism of our day, and this is just _another_ one)

Coming soon (6pm!) to the #ACL poster session: how do experts work with collections of documents, and do LLMs do those things? tl;dr: only sometimes! While we have good tools for things like information extraction, the way that experts read documents goes deeper - come to our poster to learn more!

Gently, I would like to say: When people tell you that they would appreciate a feature that does something automatically, it's not responsive to that concern to explain that by going through several steps for every individual instance, they can get the same result in each instance.

OpenAI has effectively conned people into thinking that Chatbots & AI "Assistants" are The FEWTCHA of AI. Friends, they are most likely *not.* Neither are the big cloud-based Generative AI services. Small, purpose-fit, on-device models that make your existing activities easier/better? There you go.

When it comes to text prediction, where does one LM outperform another? If you've ever worked on LM evals, you know this question is a lot more complex than it seems. In our new #acl2025 paper, we developed a method to find fine-grained differences between LMs: 🧵1/9

I for one am grateful for the opportunity to meditate on the meaning of “scientific artifact” at 2:15am

Great points in the replies here re:civility being an unequally applied, generally awful standard. It's also a great example of how common formulations of "sentiment analysis" or "toxicity detection" in NLP that are non-contextual lead to systems that stop sounding good with even slight scrutiny.

AI as another pivot to video is a thought I've also had, and this article is such a great articulation. The way "AI" is framed, pitched, and deployed by big players reflects at best ignorance, and at worst a real contempt for the social nature and humanity of our jobs and online spaces

This talk was such a joy to do! If you'd like to read the paper, it's here: arxiv.org/abs/2411.17840. Thank you for having us, @patrickbriansmith.bsky.social!

If you're at NAACL this week (or just want to keep track), I have a feed for you: bsky.app/profile/did:... Currently pulling everyone that mentions NAACL, posts a link from the ACL Anthology, or has NAACL in their username. Happy conferencing!

Ever trusted a metric that works great on average, only for it to fail in your specific use case? In our #NAACL2025 paper (w/ @841io.bsky.social), we show why global evaluations are not enough and why context matters more than you think. 📄 aclanthology.org/2025.finding... #NLP #Evaluation (🧵1/9)

🚀 Excited to share a new interp+agents paper: 🐭🐱 MICE for CATs: Model-Internal Confidence Estimation for Calibrating Agents with Tools appearing at #NAACL2025 This was work done @msftresearch.bsky.social last summer with Jason Eisner, Justin Svegliato, Ben Van Durme, Yu Su, and Sam Thomson 1/🧵

This is absurdly great, but I haven't read a single news article about it. A fully open source, offline-first alternative to Notion that's a collab between the French and German governments because they want to host docs securely and on their own terms. THIS is what Europe should be doing.

one thing in AI is not new -- people taking one small part of a job, mischaracterizing it, ignoring all the other stuff, and then assume the AI can do the whole job on its own

Have these people met … society? Read a book? Listened to music? Regurgitating esoteric facts isn’t intelligence. This is more like humanity’s last stand at jeopardy www.nytimes.com/2025/01/23/t...

[[ Now that it's not 1st anymore and more people might pay attention, sharing this again. ]] Hello *friends* doing ai + culture work, send across your burning questions about research, career, field, etc. It is anonymous :)

When I started on ARL project that funds my PhD, the thing we were supposed to build was a "MaterialsGPT". What is a MaterialsGPT? Where does that idea come from? I got to spend a lot of time thinking about that second question with @davidthewid.bsky.social and Lucy Suchman (!) working on this:

As someone whose PhD would not have been possible without military funding, this paper was a privilege to write. (Even outside of continuing to work with @davidthewid.bsky.social, and getting to work with Lucy Suchman (!))

Fully dislocated my shoulder going down some icy steps, so there will be no winter fishing for me this year :/ now gazing longingly at pictures of the last time I was out

Really interesting story! newbedfordlight.org/what-happene...

Just saw Wicked, and finally understood all of Glinda's lyrics because they were being sung by...*checks notes*...Ariana Grande???

Proposals like llms.txt (llmstxt.org) are such a fascinating rhyme of the semantic web. The standard for machine readability is so much more flexible right now, but I'd argue that we only really got here because we abandoned the original promise of upfront, user-defined machine readability!

This really changes the game for when my loved ones and I do our weekly comparison. Awful to know we've been overcompensating for frequency and length 😓

Looking for all your LTI friends on Bluesky? The LTI Starter Pack is here to help! go.bsky.app/NhTwCVb

Today is the day!! Find me at 2 PM in the Jasmine Hall (the one on the floor near food).

I've been missing paper recommendations as a regular part of my feed, so I threw together a feed that pulls every skeet (minus link repost bots) with a link to arXiv, the ACL Anthology, NeurIPS proceedings, or OpenReview: bsky.app/profile/did:... It's noisy, but maybe that's a feature?

Hell yeah 😎

I'm keeping track of people at the CMU Language Technologies Institute here: go.bsky.app/NhTwCVb. Follow along!

This was an excellent series from an excellent podcast, highly recommend!

VLADIMIR: Well? Shall we log off? ESTRAGON: Yes, let's log off. They do not log off.

me watching my advisor adjust vspace in the overleaf

"His specialty was alfalfa, and he made a good thing out of not growing any. The government paid him well for every bushel of alfalfa he did not grow. The more alfalfa he did not grow, the more money the government gave him"

Well it's called a "personal research plan" in my department

The problem isn't that AI can do your job (it can't). The problem is management thinks AI can do your job.

Me: AI assistant, decline meetings AI: meetingae, meetingās, meetingārum, meetingīs