Are there any papers which won both Best Paper award when presented and a Test of Time award later? I cannot find any in xACL. If so, suggests BP awards do not signify long-term research impact

I am becoming an editor (EiC) at TACL (Transactions of ACL) journal. Not what I expected to be doing in retirement, but I am a strong believer in journals as the best way to present research, and I want to help TACL become an even better venue for publishing NLP research.

INLG (@inlg.bsky.social) submissions are now open! Please submit all of your work on Natural Language Generation. Submit systems, demos, experiments, position papers, squibs, linguistic analyses, as long as it is NLG-related. For more information, see: 2026.inlgmeeting.org #NLProc

Really interesting survey about peer review www.cs.cmu.edu/~nihars/prep... (extended version of CACM paper). Lots of interesting and indeed worrying observations about peer review

New blog: Future of NLG evaluation In a recent position paper, I argued that NLG evaluation in the future needs to be become more rigorous. It also needs to move beyond benchmarks, and focus more on impact, qualitative, and safety evaluation. ehudreiter.com/2026/06/26/f...

My student @iniakpothompson.bsky.social developed a safe driving app for Nigeria. Evaluation was promising, and he is currently trying to convert his research project into a reality which saves lives in Nigeria. If anyone wants to learn more about this work, see iniakpothompson.com/research-pro...

Really enjoyed helping my colleague Jakub Zbrzezny from our Divinity dept look at how well LLMs can translate biblical materials in a local Arabic dialect. Not surprisingly, LLMs good at translating out of dialect, but struggle to translate into dialect. aclanthology.org/2026.retroev...

Radical suggestion. Why not *lower* prestige of xACL, for example by including Findings in main conf. Then people chasing N papers in "top" venues will submit elsewhere, making xACL more manageable. Let Neurips deal with AI slop...

I am pleased to announce that the proceedings for RetroEval have now been published on ACL Anthology: aclanthology.org/volumes/2026...

Reviewed a paper which extensively used a dataset I helped to create, but showed zero awareness of data issues and the domain. I guess people just want data to throw into LLMs, dont care about data issues even when these are carefully explained in dataset paper.

[AI leads to] an expansion of individual scientists’ impact but a contraction in collective science’s reach, as AI-augmented work moves collectively towards areas richest in data... AI tools seem to automate established fields rather than explore new ones www.nature.com/articles/s41...

We had a fantastic time last week discussing the current challenges in NLG evaluation and celebrating the career of @ehudreiter.bsky.social. Pictures and a few video clips are now available: retroeval.github.io/_pages/media I would like to thank @uniofaberdeen.bsky.social for their support.

New blog: I am worried by NLP research culture NLG and NLP are mostly much better in 2026 than when I got my PhD in 1990. Unfortunately research culture has gotten *worse” in this period, which really worries me as I retire. ehudreiter.com/2026/06/08/n...

On Tuesday at the #RetroEval symposium the SIGGEN board awarded Ehud the Inaugural Prize for Outstanding Contributions to Natural Language Generation, future editions of which will bear his name as the Ehud Reiter Prize for Outstanding Contributions to Natural Language Generation. Thank you, Ehud!

I felt honoured and was very happy to get an Outstanding Contribution in NLG award from @siggen.bsky.social !

Took lots of noce pictures at my Retroeval retirement symposium. One of my favourites was me with my first and last PhD students, Sandra Williams and Yujun Wang!

I'm looking forward to my retirement workshop, which starts on Monday 1 June!! Will be great to catch up with former students and colleagues, and also discuss NLG evaluation. retroeval.github.io

I remember seeing very dubious advice from OpenAI a few years ago on evaluation. So I was happy to see quite sensible recent advice from Anthropic on evaluation www.anthropic.com/engineering/...

Really interesting scoping review that points out numerous flaws in LLM-as-Judge evaluation in healthcare, including minimal human oversight, absent bias testing, model monoculture, ignore implicit eval components, no check for consistency over time (etc) arxiv.org/abs/2604.25933

Someone asked me what were the highlights of my career, I responded with a list of papers which I was proud of. I did not mention grants, awards, jobs, etc. I know some people are proudest of their grants (etc), but for me it was always scientific outputs.

I wrote paper on "NLG Evaluation: Past, Present, Future" for Retroeval. Eval has changed enornously over my career! In future, I expect more on stuff relevant to real-world usage, including impact, qualitative studies, safety in worst/adversarial case arxiv.org/abs/2605.23715

New blog: Software engineering of prompts Creating complex prompts for LLMs faces similar software engineering challenges as conventional software (requirements, design, testing, maintenance). We need to understand good software engineering for prompts. ehudreiter.com/2026/05/20/s...

Congrats to my student Jawwad Baig for passing his PhD viva! Topic was “Data-to-Text NLG Feedback for Safer Driving”. Jawwad did his PhD part-time (ie, evenings and weekends while he worked fulltime) and remote (lives in England), which is very tough, but he still completed

I think we need meaningful penalties on fraudulent academic papers. Glad to see Arxiv is taking action!

ArXiV has a new LLM policy (Screenshots with alt text so you don’t have to click through to the other place and see all the stupid responses)

Have now resigned as ARR (meta-)reviewer. I will continue to do some reviewing after retirement, but not ARR. I dont think mega-conf are the right way to present important research findings, and ARR reviewing is not enjoyable, eg I have no control over what I am asked to review

I am happy to announce that registration for RetroEval 2026 (retroeval.github.io) is now open! Additionally, the program is now available including keynote speakers and accepted papers: retroeval.github.io/_pages/progr... Looking forward to seeing you at the @uniofaberdeen.bsky.social in June!

The International conference on Natural Language Generation (INLG) just issued the first Call For Papers: 2026.inlgmeeting.org/calls.html #nlproc

Asked about students cheating in CS using AI. Said I was not concerned about cheating distorting marks, but was very concerned that it demotivated students from learning. I gave assess which AI cannot do, failure rate skyrocketed compared to prev year ehudreiter.com/2026/05/05/a...

How will AI impact CS teaching? Biggest challenge is adapting what we teach to a world where AI assistants are heavily used. We should also use AI tutors. Least important is making assessments more resistant to AI cheating ehudreiter.com/2026/05/05/a...

What % of the NLP papers measure their impact in the real world? This paper proposes an impact evaluation of NLP models or systems for real-world usage, changing the research culture of NLP to focus more on real-world impact and less on SOTA-chasing: doi.org/10.1162/COLI... @ehudreiter.bsky.social

We’ve opened up pre-registration for the Symposium on Natural Language Generation Evaluations (retroeval.github.io/index). Come join us in Aberdeen for two days of deep discussions on all things evaluation and celebrate the career of Prof. @ehudreiter.bsky.social: forms.office.com/e/ZbUmxGraAS

Our final year UG students turn in their honours projects today. Supervising projects is the nicest part of teaching for me - always learning something, and great to supervise students 1-1. Really nice projects this year on evaluating LLM in real-world, and digital humanities.

Nice "end-of-teaching" event yesterday, which included past MSc students saying how much they learned from my classes. Always nice to get positive feedback!

My student Adarsa Sivaprasad is looking for people who have lived experience of IVF, to help in evaluating a new AI chatbot which helps people understand IVF outcome predictions. qfreeaccountssjc1.az1.qualtrics.com/jfe/form/SV_...

My PhD students gave me below to mark the end of my teaching career. Love it!

Gave my last lecture and practical lab before retirement, on NLG evaluation to our MSc students. A bit anticlimatic, since attendance and engagement was poor, maybe because some students are relying on ChatGPT. Perhaps this is a good time to retire from CS teaching...

I am frequently told how amazing LLMs are, but they cannot do what I want them to do. They make too many mistakes. Theyt also they lack solid evidence of effectiveness and require too much resource. ehudreiter.com/2026/04/20/a...

If you're working on your submission to #RetroEval2026, the Symposium on Natural Language Generation Evaluations in honor of @ehudreiter.bsky.social, I've got some good news! The deadline has been extended to 28 April for archival submissions! Updated deadlines, etc, here: retroeval.github.io

New blog: Achieving my vision of personal AI health assistants My 25-year old vision of personal health assistants is closer to reality, but challenges remain including requirements, adapting to individuals, RCTs, running on cheap phones. ehudreiter.com/2026/04/20/a...

Asked Google and ChatGPT about size of the Aberdeen Jewish community, got (confident) wrong answers from both (I am president of Aber synagogue). Because both rely on very outdated websites. Even best tech gives bad answers if it relies on low-quality out-of-date Internet data...

As I watch my students do prompt engineering, feels they are trying to write a program in a very high-level language where they cannot predict the effects of changes to the program, cannot guarantee stability as models improve, and have little confidence in robustness in handling unseen dta. Hum...

New paper on LLMs and research methodology: Justify your prompts! direct.mit.edu/coli/article... #nlproc

Congatulations to Iniakpokeikiye Thompson, who passed his PhD viva yesterday with minor corrections! Thompson's thesis is on developing an app to encourage safer driving in Nigeria. Really interesting to see the differences when doing this in Nigeria instead of UK. ehudreiter.com/2025/09/03/e...

New blog: Real-world safety and harms from patient-facing LLMs There is limited data on harms to real patients from AI chatbots. Incident reports, clinical trials, and health provider data suggest bots are usually safe, but can be harmful in a few cases. ehudreiter.com/2026/04/06/r...

Seen several new papers criticising performance benchmarks. Why is anyone surprised? No one cares whether performance benchmarks mean anything - vendors see them as marketing tools, and academics just want to beat something from a SOTA perspective. Neither cares about validity

📍It's official! #INLG2026 is coming to Utrecht, Netherlands, Oct 17-21! Hosted with support from Utrecht University and the local NLP community. Follow us here and check 2026.inlgmeeting.org for updates -- hope to see you there!

Are there papers/data on real-world safety problems and harms to patients from using AI chatbots for health? I've seen papers which ask patients to role play scenarios, but is there anything where people ask about their own health? Surveys, incident reporting, or trials of bots in live usage?

Noce paper on how well benchmarks cover what people do at work arxiv.org/abs/2603.01203 Quote "these observations suggest that agent benchmarking effort is driven less by alignment with real-world employment structure or economic value, and more by methodological convenience."

New blog: Comparing performance of LLMs is not very interesting Comparing different LLMs in research papers may not mean much, because LLMs will be out of date when paper published. Finding behaviour which is shared by LLMs is interesting and worthwhile. ehudreiter.com/2026/03/24/c...