In a new blog post, I argue that the anti-ai movement ought to distinguish between claims about the technology and the "project of AI," as defined by Vetsi et al. in their new paper. 🔗: doomscrollingbabel.manoel.xyz/p/the-anti-a...

Any papers come to mind as really good examples of qualitative evaluation/analysis in NLP?

The International conference on Natural Language Generation (INLG) just issued the first Call For Papers: 2026.inlgmeeting.org/calls.html #nlproc

Yes, though it’s kind of difficult to avoid this in our field. These intuitive terms catch on too easily and are almost impossible to replace once they are established.

Not sure how I missed this paper.ore relevant than ever now. aclanthology.org/2023.emnlp-m...

What % of the NLP papers measure their impact in the real world? This paper proposes an impact evaluation of NLP models or systems for real-world usage, changing the research culture of NLP to focus more on real-world impact and less on SOTA-chasing: doi.org/10.1162/COLI... @ehudreiter.bsky.social

New experimental philosophy paper by Eugen Fischer on the ordinary understanding of perception Philosophers have long debated two opposing views: direct realism and indirect realism. Which do people think is right? The key result: Many people endorse BOTH link.springer.com/article/10.1...

When does using AI to produce research slip into fraud? And why does so much AI-for-science 'optimism' actually rest on a deeply pessimistic view of scientific agency? 🤔 New post: statmodeling.stat.columbia.edu/2026/04/22/f...

In a post-CHI blog post, I talk about what I believe technology like the Anthropic Interviewer will mean for qualitative research. doomscrollingbabel.manoel.xyz/p/qualitativ...

New paper on LLMs and research methodology: Justify your prompts! direct.mit.edu/coli/article... #nlproc

Happy to see this accepted at #ACL2026 main!

Is a study still qualitative if humans don't conduct the interviews or do the analysis? 🤔

My #atscience talk about Lea is now online! If you've only ever heard me talk about NLP, here's a chance to hear me rant about #atproto (the protocol you're reading this on!), social media for researchers, and the free internet. And if you're interested in helping us build Lea, please reach out!

📍It's official! #INLG2026 is coming to Utrecht, Netherlands, Oct 17-21! Hosted with support from Utrecht University and the local NLP community. Follow us here and check 2026.inlgmeeting.org for updates -- hope to see you there!

If you’re at #EACL2026 today, stop by @zdenekkasner.cz at the MME workshop to chat about our work on span annotation! Paper: aclanthology.org/2026.mme-mai...

New preprint: arxiv.org/abs/2603.20133 Does performance on reasoning benchmarks transfer to real-world settings such as task-oriented dialogue? Not necessarily: our new benchmark tests LLMs on problems framed in both standalone and dialogue settings, and shows that dialogue makes reasoning harder.

Short musings on "cognitive debt" - I'm seeing this in my own work, where excessive unreviewed AI-generated code leads me to lose a firm mental model of what I've built, which then makes it harder to confidently make future decisions simonwillison.net/2026/Feb/15/...

The 5th Generation, Evaluation, and Metrics (GEM) Workshop will be at #ACL2026! Call for papers is out. Topics include: 🐟 LMs as evaluators 🐠 Living benchmarks 🍣 Eval with humans and more New for 2026: Opinion & Statement Papers! Full CFP: gem-workshop.com/call-for-pap...

My latest on Substack -- a write-up of the talk I gave at NeurIPS in December. aiguide.substack.com/p/on-evaluat...

Our paper "OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMs" has been accepted to #INLG2025 conference! You can read the preprint here: arxiv.org/abs/2503.11858

#ACL2025NLP in Vienna 🇦🇹 starts today with 23 🤯 @ufal-cuni.bsky.social folks presenting their work both at the main conference and workshops. Check out our main conference papers today and on Wednesday 👇

Slides and links to papers at bit.ly/mlprague25-od 🤓

Today, @tuetschek.bsky.social shared the work of his team on evaluating LLM text generation with both human annotation frameworks and LLM-based metrics. Their approach tackles the benchmark data leakage problem and how to get unseen data for unbiased LLM testing.

How do LLMs compare to human crowdworkers in annotating text spans? 🧑🤖 And how can span annotation help us with evaluating texts? Find out in our new paper: llm-span-annotators.github.io Arxiv: arxiv.org/abs/2504.08697