🏆🪩 We just won the best paper award at ICLR'26-CAO with DISCO! I am so proud of my co-authors! Huge congrats to @arubique.bsky.social, Benjamin, and @coallaoh.bsky.social

1/ My contract at @parameterlab.bsky.social ended last week, after 2.5 years (since Sept 2023, with some collaboration before). I had the chance to lead research on trustworthy AI for LLMs alongside an incredible group of people. (Neckarfront. All Tübingen researcher have to post it once!)

🎉 Our privacy collapse paper has been accepted at #ACL 2026 (main)! Contextual privacy is fragile: fine-tune an LLM on benign data, and it can overshare personal information. This is silent. Safety suites don't measure contextual privacy, which is a problem now that most applications are agentic.

🌍 We've made LLM watermarking equally robust across all languages we studied, while scaling to 100+ languages! Even sota watermarks can be removed by translating to another language, eg. Tamil. This hits hardest in low-resource languages, where moderation tools are already weak. 🧵

The D&B track now has a larger scope and a new name: Evaluation & Datasets. It focuses on evaluation itself as a scientific object. It is really nice to have somewhere for critical analysis of evaluation and negative results. It was really missing in ML!

NeurIPS deadline is out! Add the 6th of May to your calendar :)

LLM agents include far more than a model: framework, orchestration, tools, error handling, etc. These harness engineering choices matter, but they're rarely compared. MASEval makes that straightforward. I'm very proud to have supervised its development. Give it a look! ⬇️

If you want to get up to speed on what all the benchmarks mean, I wrote a bunch of digests for the popular ones over on the ngrok blog. Designed for people that are interested but not enough to go read all the papers. ngrok.com/blog/ai-benc...

New paper out!🎉 One of our most surprising findings: fine-tuning an LLM on debugging code has unexpected side-effects on contextual privacy. The model learns from printing variables that internal state are ok to share, then generalises this to social situations🤯 A🧵below👇

🎉Thrilled to share that both of my #ICLR2026 submissions were accepted (2/2)! 🪩 DISCO, Efficient Benchmarking: bsky.app/profile/arub... 🩺 Dr.LLM, Dynamic Layer Routing: www.linkedin.com/posts/ahmed-... Huge thanks to my co-authors, especially first authors @arubique.bsky.social & Ahmed Heakl!

🧵 Many hidden gems about LLM benchmark contamination in the GAPERON paper! This French-English model paper has some honest findings about how contamination affects benchmarks (and why no one wants to truly decontaminate their training data) Thread 👇

Delighted to announce that 3.5 years after my first first-author paper was accepted at UAI 2022, I've been appointed Area Chair for UAI 2026! 😊 UAI was my first in-person conference right after COVID 1/2

Our #EMNLP2025 paper Leaky Thoughts 🫗 shows that Large Reasoning Models (LRMs) can unintentionally leak sensitive information hidden in their internal thoughts. 📍 Come chat with Tommaso at our poster on Friday 7th, 10:30–12:00 in Hall C3 📄 aclanthology.org/2025.emnlp-m...

🪩 New paper out! Evaluating large models on benchmarks like MMLU is expensive. DISCO cuts costs by up to 99% while still predicting well performance. 🔍 The trick: use a small subset of samples where models disagree the most. These are the most informative. Join the dance party below 👇

🎉 Delighted to announce that our 🫗Leaky Thoughts paper about contextual privacy with reasoning models is accepted to #EMNLP main! Huge congrats to the amazing team Tommaso Green, Haritz Puerto @coallaoh.bsky.social @oodgnas.bsky.social

Fantastic new paper by @reeserichardson.bsky.social et al. An enormous amount of work showing the extent of coordinated scientific fraud and involvement of some editors. The number of fraudulent publications grows at a rate far outpacing that of legitimate science. www.pnas.org/doi/10.1073/...

📢 New paper out: Does SEO work for LLM-based conversational search? We introduce C-SEO Bench, a benchmark to test if conversational SEO methods actually help. Our finding? They don't. But traditional SEO still works because LLMs favour content already ranked higher in the prompt.

The mood on a Friday evening

Excited to share that our paper "Scaling Up Membership Inference: When and How Attacks Succeed on LLMs" will be presented next week at #NAACL2025! 🖼️ Catch us at Poster Session 8 - APP: NLP Applications 🗓️ May 2, 11:00 AM - 12:30 PM 🗺️ Hall 3 Hope to see you there!

A Bluesky filter to recommend only posts about papers from your followers. This is what I was missing to use Bluesky!

I am pleased to announce that our paper on the scale of LLM membership inference from @parameterlab.bsky.social has been accepted for publication at #NAACL2025 as Findings!

🎉We’re pleased to share the release of the models from our Apricot🍑 paper, accepted at ACL 2024! At Parameter Lab, we believe openness and reproducibility are essential for advancing science, and we've put in our best effort to ensure it. 🤗 huggingface.co/collections/... 🧵 bsky.app/profile/dnns...

📄 Excited to share our latest paper on the scale required for successful membership inference in LLMs! We investigate a continuum from single sentences to large document collections. Huge thanks to an incredible team: Haritz Puerto, @coallaoh.bsky.social and @oodgnas.bsky.social!

Have a look at the 🍑 Apricot paper that we presented at ACL earlier this year. This project was a wonderful collaboration with @dnnslmr.bsky.social!

After going to NAACL, ACL and #EMNLP2024 this year, here are a few tips I’ve picked up about attending #NLP conferences. Would love to hear any other tips if you have them! This proved very popular on another (more evil) social media platform, so sharing here also 🙂 My 10 tips:

🌟 Pleased to join Bluesky! As a first post, allow me to share my latest first-author paper, TRAP 🪤, presented at #ACL24 (findings). 🦹💥 We explore how to detect if an LLM was stolen or leaked🤖💥 We showcase how to use adversarial prompt as #fingerprint for #LLM. A thread 🧵 ⬇️⬇️⬇️