I am hiring PhD students and/or Postdocs, to work on the theory of explainable machine learning. Please apply through Ellis or IMPRS, deadlines end october/mid november. In particular: Women, where are you? Our community needs you!!! imprs.is.mpg.de/application ellis.eu/news/ellis-p...

We need new rules for publishing AI-generated research. The teams developing automated AI scientists have customarily submitted their papers to standard refereed venues (journals and conferences) and to arXiv. Often, acceptance has been treated as the dependent variable. 1/

This new center strikes the right tone in approaching the AI alignment problem. alignmentalignment.ai

A new recording of our FridayTalks@Tübingen series is online! How much can we forget about Data Contamination? by @sbordt.bsky.social Watch here: youtu.be/T9Y5-rngOLg

The stochastic parrot is now an IMO gold medalist parrot

I'm at #ICML in Vancouver this week, hit me up if you want to chat about pre-training experiments or explainable machine learning. You can find me at these posters: Tuesday: How Much Can We Forget about Data Contamination? icml.cc/virtual/2025...

Our #ICML position paper: #XAI is similar to applied statistics: it uses summary statistics in an attempt to answer real world questions. But authors need to state how concretely (!) their XAI statistics contributes to answer which concrete (!) question! arxiv.org/abs/2402.02870

During the last couple of years, we have read a lot of papers on explainability and often felt that something was fundamentally missing🤔 This led us to write a position paper (accepted at #ICML2025) that attempts to identify the problem and to propose a solution. arxiv.org/abs/2402.02870 👇🧵

Have you ever wondered whether a few times of data contamination really lead to benchmark overfitting?🤔 Then our latest #ICML paper about the effect of data contamination on LLM evals might be for you!🚀 Paper: arxiv.org/abs/2410.03249 👇🧵

In explainable machine learning, we mostly have negative results for what post-hoc explanations cannot do. This work presents a surprisingly strong positive result for SHAP, showing that a simple sampling modification allows to reliably detect features that don't influence the model.

Finally made it to bluesky as well ...

Is the distinction between "aleatoric" and "epistemic" uncertainty practically meaningful (or even well defined) in any real sense? Aleotoric uncertainty refers to irreducible unpredictability (e.g. unrealized randomness in nature) whereas epistemic refers to model uncertainty.

I really like coding with LLMs. This week Claude & ChatGPT convinced me my code was too slow. After 2 days of investigation, I think my code is just fine. Never again will I blindly trust you with my profiler logs!🤖

I really like the new HTML preview on arxiv, but it somehow handles latex errors differently from PDF. I've been seeing lots of ICML error messages lately.

can you draw me a dragon in tikz

I just asked aistudio.google.com to write a review for a paper that we will submit to ICML. It's impressive. I believe with this tool, I could produce a mediocre paper review for almost any paper in less than 10 minutes (judged by the standard of reviews that we have at ML conferences).

Uhm so OpenAI actually has access to FrontierMath? epoch.ai/blog/openai-...

The chain of thought in DeepSeek-R1 is pretty impressive.

ICML 2025 has some exciting changes. Here are two of my favorites. 1. Only 1 round of back-and-forth between authors & reviewers. The review process should not be an endless back and forth. It shouldn't be possible to get your paper accepted by exhausting reviewers.

Are you interested in data contamination and LLM benchmarks?🤖 Check out our poster today at the NeurIPS ATTRIB workshop (3-4:30pm)! 💡 TL;DR: In the large-data regime, a few times of data contamination matter less than you might think.

Recent works have proposed to use publicly available Kaggle competitions to benchmark LLMs, most famously OpenAI's openai.com/index/mle-be.... In this blog, I show how to test LLMs for contamination with Kaggle competitions (of course, there is contamination). sbordt.substack.com/p/data-conta...

thinking of calling this "The Illusion Illusion" (more examples below)

Check out this starter pack! go.bsky.app/BYkRryU

Here's a fledgling starter pack for the AI community in Tübingen. Let me know if you'd like to be added! go.bsky.app/NFbVzrA

Recent works have proposed to use publicly available Kaggle competitions to benchmark LLMs, most famously OpenAI's openai.com/index/mle-be.... In this blog, I show how to test LLMs for contamination with Kaggle competitions (of course, there is contamination). sbordt.substack.com/p/data-conta...