Why is the default option in the rebuttal acknowledgements at ICML to accept the paper? I'm very confused on how to use these buttons. Can someone explain them to me?

Neural networks are highly non-convex, so approximate error minimizers need not look anything like each other in parameter space. But we show that nevertheless (for many model sizes) approximate error minimizers must closely agree in function/prediction space despite this!

It's this time of the year again: your baselines cannot be PPO and SAC.

Michael @mkearnsphilly.bsky.social ) and I wrote a blog post about our experiences using AI for research, and our thoughts on what these developments will mean for research, publication, and education: www.amazon.science/blog/how-ai-...

I'm so glad that so many research problems are finally being treated as first class citizens rather than afterthoughts. 🤔

Too many papers sound like this Hierarchical Context-Aware Diffusion-Transformer Meta-World-Model Reinforcement Learning with Causally Disentangled Preference-Aligned Self-Supervised Compositional Multi-Scale Latent Skill Priors for Long-Horizon Generalist Decision Making

One reason I work on replicable and consistent RL is because it is has always been at the top of the list of criteria for reliability.

Excuse me? Surely telling me that didn't require much thinking.

I have seen multiple times now that a reviewer said sth like: the proofs are simple -> reject the paper. That is completely counter-productive. A theorem needs to generate new insights. If we learn something new from something simple that should be preferred. Don't believe me? Ask someone famous:

I’ve been thinking about a practical question and would love some opinions: How do your papers actually get discovered/cited? I was searching for recent work on high update ratio RL and found several very closely related papers tackling the same failure modes we study. None cited our earlier work.

🚀 Excited to share REPPO, a new on-policy RL agent! TL;DR: Replace PPO with REPPO for fewer hyperparameter headaches and more robust training. REPPO, led by @cvoelcker.bsky.social, will be presented at ICLR 2026. How does it work? 🧵👇

Scaling Laws in Particle Physics Data! This is a result I've been itching to share and it's finally out. One of the big open questions is how much better AI-based methods at particle colliders can still become. 1/4

"Scientific reviewers should have experience publishing scientific work in related areas" is really not that hot of a take.

Clicking like on any relevant ICLR paper. Encourage people to post their work here more!

The other paper accepted to @iclr-conf.bsky.social 2026 🇧🇷. Our work on replicable RL sheds some light on how to consistently make decisions in RL. @ericeaton.bsky.social @mkearnsphilly.bsky.social @aaroth.bsky.social @sikatasengupta.bsky.social @optimistsinc.bsky.social

Two papers accepted to @iclr-conf.bsky.social 2026! One of the is REPPO, see below! I think it deserves a lot more recognition. Let's chat about it in Rio! 🇧🇷

Quite disheartening that there isn't a single workshop at ICLR to present my RL work but there several topics that are listed 5 or 6 times just named differently.

Our number went down by 0.01 but it's very expensive to run so we can't have error bars. Our algorithm is so much better than the rest, new SOTA!

I can't believe that this paper is not yet used by literally everyone. Claas doing all he can to make your life easier. Check it out.

Bringing this back up.

Excited about a new paper! Multicalibration turns out to be strictly harder than marginal calibration. We prove tight Omega(T^{2/3}) lower bounds for online multicalibration, separating it from online marginal calibration for which better rates were recently discovered.

In emergencies, minutes can decide who lives and who dies. Our team is participating in the Triage Challenge, building AI to empower clinicians and prioritize care when resources are thin: prontotriage.com Also recently featured by Meta: ai.meta.com/blog/upenn-d... (1/5)

Posted about this last week; I feel like it didn't get as much attention as it deserves. We have a new preprint on using diffusion models to generate compositional data. This work was conducted by Quan, a student I advised over the fall. He is currently looking for PhD positions. Check it out!

We have a new preprint out on iterative generation of compositional robot data. This work was conducted by Quan, one of the students I advised over the semester. Check out the thread! P.S. Quan is currently looking for PhD positions, keep an eye out for him!

Getting ready to leave for NeurIPS tomorrow morning. ✈️ Let me know if you're around and wanna hang out!

Me all day today

It's mind-boggling to my how many of the papers I reviewed don't cite a single paper that is older than like 2020. It's like we try to collectively forget what people did in the past so we can publish more.

Am at Columbia today giving a talk about (in part) this work. Have a few hours to kill afterwards. If anyone is around there and wants to chat, DM me.

I think I posted about it before but never with a thread. We recently put a new preprint on arxiv. 📖 Replicable Reinforcement Learning with Linear Function Approximation 🔗 arxiv.org/abs/2509.08660 In this paper, we study formal replicability in RL with linear function approximation. The... (1/6)

I have been told I need to get more modern in my paper promotion! github.com/cvoelcker/reppo / arxiv.org/abs/2507.11019 @marcelhussing.bsky.social

Super stoked for the New York RL workshop tomorrow. Will be presenting 2 orals: * Replicable Reinforcement Learning with Linear Function Approximation * Relative Entropy Pathwise Policy Optimization We already posted about the 2nd one (below), I'll get to talking about the first one in a bit here.

(Maybe) unpopular opinion: There should not be *any* new experiments in a rebuttal. A rebuttal is for clarifications and incorrect statements in a review. You should not be allowed to add new content at that point. Either your paper is done or it isn't. It should not be written during rebuttals.

New ChatGPT data just dropped

My PhD journey started with me fine-tuning hparams of PPO which ultimately led to my research on stability. With REPPO, we've made a huge step in the right direction. Stable learning, no tuning on a new benchmark, amazing performance. REPPO has the potential to be the PPO killer we all waited for.

🔥 Presenting Relative Entropy Pathwise Policy Optimization #REPPO 🔥 Off-policy #RL (eg #TD3) trains by differentiating a critic, while on-policy #RL (eg #PPO) uses Monte-Carlo gradients. But is that necessary? Turns out: No! We show how to get critic gradients on-policy. arxiv.org/abs/2507.11019

Works that use #VAML/ #MuZero losses often use deterministic models. But if we want to use stochastic models to measure uncertainty or because we want to leverage current SOTA models such as #transformers and #diffusion, we need to take care! Naively translating the loss functions leads to mistakes!

Dhruv Rohatgi will be giving a lecture on our recent work on comp-stat tradeoffs in next-token prediction at the RL Theory virtual seminar series (rl-theory.bsky.social) tomorrow at 2pm EST! Should be a fun talk---come check it out!!

Just arrived in Montreal for my internship at FAIR. So far Montreal has been amazing, great walkable areas, good food and nice people! Although I must say I have to get used to being addressed in French 😅

We'll be presenting our work on Oracle-Efficient Reinforcement Learning for Max Value Ensembles at the RL theory seminar! Been following this series for a while, super excited we get to present some of our work. 🥳

Many great papers from Mila! Two by my team at the Adaptive Agents Lab (Adage) together with collaborators: A Truncated Newton Method for Optimal Transport openreview.net/forum?id=gWr... MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL openreview.net/forum?id=6Rt... #ICLR2025

📢 Deadline Extension Alert! 📢 Good news! We’re extending the #CoLLAs2025 submission deadlines: 📝 Abstracts: Feb 26, 2025, 23:59 AoE 📄 Papers: Mar 3, 2025, 23:59 AoE More time to refine your work—don't miss this chance to contribute to #lifelong-learning research! 🚀 🔗 lifelong-ml.cc

I was very hyped about this place initially, now I come here, see 5 posts about politics, unfollow 5 people and close the website. Where are the interesting AI posts?

Can you solve group-conditional online conformal prediction with a no-regret learning algorithm? Not with vanilla regret, but -yes- with swap regret. And algorithms from the follow-the-regularized leader family (notably online gradient descent) work really well for other reasons.

Bummed out about recent politics & news drowning out AI and science you want to see on Bluesky? Well, here is a small "sky thread" (written on a ✈️) about something I recently discovered: e-values! They are an alternative to the standard p-values as a measure of statistical significance. 1/N

Throwing compute at things has proven quite powerful in other domains but until recently not as much in #ReinforcementLearning. Excited to share that out MAD-TD paper got a spotlight at #ICLR25! Check out Claas' thread on how to get the most out of your compute/data buck when training from scratch.

EC 2025 (S)PC --- lets get ready for the Super Bowl! Every time there is a first down, bid on a paper. Field goal? Bid on two. Touchdown? Bid on 5 papers (10 if its the Eagles!) At the halftime show enter your topic preferences and conflicts. Lets go birds!

🚨🚨 RLC deadline has been extended by a week! Abstract deadline is Feb. 21 with a paper deadline of Feb. 28 🚨🚨. Please spread the word!

What a future work section should be: Oh, and here is this interesting and hard open problem that someone should solve. Future work sections in empirical ML papers: We leave hyperparameter optimization for future work.

My new year's resolution is to spend more time thinking. Last year I found myself deep in the nitty gritty of creating solutions. While that is important it is also necessary to reflect and look at the bigger picture. Entering my 5th year, I will try to focus more on defining the next problems.