Our number went down by 0.01 but it's very expensive to run so we can't have error bars. Our algorithm is so much better than the rest, new SOTA!

I can't believe that this paper is not yet used by literally everyone. Claas doing all he can to make your life easier. Check it out.

Bringing this back up.

Excited about a new paper! Multicalibration turns out to be strictly harder than marginal calibration. We prove tight Omega(T^{2/3}) lower bounds for online multicalibration, separating it from online marginal calibration for which better rates were recently discovered.

In emergencies, minutes can decide who lives and who dies. Our team is participating in the Triage Challenge, building AI to empower clinicians and prioritize care when resources are thin: prontotriage.com Also recently featured by Meta: ai.meta.com/blog/upenn-d... (1/5)

Posted about this last week; I feel like it didn't get as much attention as it deserves. We have a new preprint on using diffusion models to generate compositional data. This work was conducted by Quan, a student I advised over the fall. He is currently looking for PhD positions. Check it out!

We have a new preprint out on iterative generation of compositional robot data. This work was conducted by Quan, one of the students I advised over the semester. Check out the thread! P.S. Quan is currently looking for PhD positions, keep an eye out for him!

Getting ready to leave for NeurIPS tomorrow morning. ✈️ Let me know if you're around and wanna hang out!

Me all day today

It's mind-boggling to my how many of the papers I reviewed don't cite a single paper that is older than like 2020. It's like we try to collectively forget what people did in the past so we can publish more.

Am at Columbia today giving a talk about (in part) this work. Have a few hours to kill afterwards. If anyone is around there and wants to chat, DM me.

I think I posted about it before but never with a thread. We recently put a new preprint on arxiv. 📖 Replicable Reinforcement Learning with Linear Function Approximation 🔗 arxiv.org/abs/2509.08660 In this paper, we study formal replicability in RL with linear function approximation. The... (1/6)

I have been told I need to get more modern in my paper promotion! github.com/cvoelcker/reppo / arxiv.org/abs/2507.11019 @marcelhussing.bsky.social

Super stoked for the New York RL workshop tomorrow. Will be presenting 2 orals: * Replicable Reinforcement Learning with Linear Function Approximation * Relative Entropy Pathwise Policy Optimization We already posted about the 2nd one (below), I'll get to talking about the first one in a bit here.

(Maybe) unpopular opinion: There should not be *any* new experiments in a rebuttal. A rebuttal is for clarifications and incorrect statements in a review. You should not be allowed to add new content at that point. Either your paper is done or it isn't. It should not be written during rebuttals.

New ChatGPT data just dropped

My PhD journey started with me fine-tuning hparams of PPO which ultimately led to my research on stability. With REPPO, we've made a huge step in the right direction. Stable learning, no tuning on a new benchmark, amazing performance. REPPO has the potential to be the PPO killer we all waited for.

🔥 Presenting Relative Entropy Pathwise Policy Optimization #REPPO 🔥 Off-policy #RL (eg #TD3) trains by differentiating a critic, while on-policy #RL (eg #PPO) uses Monte-Carlo gradients. But is that necessary? Turns out: No! We show how to get critic gradients on-policy. arxiv.org/abs/2507.11019

Works that use #VAML/ #MuZero losses often use deterministic models. But if we want to use stochastic models to measure uncertainty or because we want to leverage current SOTA models such as #transformers and #diffusion, we need to take care! Naively translating the loss functions leads to mistakes!

Dhruv Rohatgi will be giving a lecture on our recent work on comp-stat tradeoffs in next-token prediction at the RL Theory virtual seminar series (rl-theory.bsky.social) tomorrow at 2pm EST! Should be a fun talk---come check it out!!

Just arrived in Montreal for my internship at FAIR. So far Montreal has been amazing, great walkable areas, good food and nice people! Although I must say I have to get used to being addressed in French 😅

We'll be presenting our work on Oracle-Efficient Reinforcement Learning for Max Value Ensembles at the RL theory seminar! Been following this series for a while, super excited we get to present some of our work. 🥳

Many great papers from Mila! Two by my team at the Adaptive Agents Lab (Adage) together with collaborators: A Truncated Newton Method for Optimal Transport openreview.net/forum?id=gWr... MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL openreview.net/forum?id=6Rt... #ICLR2025

📢 Deadline Extension Alert! 📢 Good news! We’re extending the #CoLLAs2025 submission deadlines: 📝 Abstracts: Feb 26, 2025, 23:59 AoE 📄 Papers: Mar 3, 2025, 23:59 AoE More time to refine your work—don't miss this chance to contribute to #lifelong-learning research! 🚀 🔗 lifelong-ml.cc

I was very hyped about this place initially, now I come here, see 5 posts about politics, unfollow 5 people and close the website. Where are the interesting AI posts?

Can you solve group-conditional online conformal prediction with a no-regret learning algorithm? Not with vanilla regret, but -yes- with swap regret. And algorithms from the follow-the-regularized leader family (notably online gradient descent) work really well for other reasons.

Bummed out about recent politics & news drowning out AI and science you want to see on Bluesky? Well, here is a small "sky thread" (written on a ✈️) about something I recently discovered: e-values! They are an alternative to the standard p-values as a measure of statistical significance. 1/N

Throwing compute at things has proven quite powerful in other domains but until recently not as much in #ReinforcementLearning. Excited to share that out MAD-TD paper got a spotlight at #ICLR25! Check out Claas' thread on how to get the most out of your compute/data buck when training from scratch.

EC 2025 (S)PC --- lets get ready for the Super Bowl! Every time there is a first down, bid on a paper. Field goal? Bid on two. Touchdown? Bid on 5 papers (10 if its the Eagles!) At the halftime show enter your topic preferences and conflicts. Lets go birds!

🚨🚨 RLC deadline has been extended by a week! Abstract deadline is Feb. 21 with a paper deadline of Feb. 28 🚨🚨. Please spread the word!

What a future work section should be: Oh, and here is this interesting and hard open problem that someone should solve. Future work sections in empirical ML papers: We leave hyperparameter optimization for future work.

My new year's resolution is to spend more time thinking. Last year I found myself deep in the nitty gritty of creating solutions. While that is important it is also necessary to reflect and look at the bigger picture. Entering my 5th year, I will try to focus more on defining the next problems.

Apparently I'm in the top 1% of wandb users. Good or bad sign?

Reposting a postdoc opportunity in field robotics at Penn, in case anyone missed it the first time around.

Microsoft's Computational Social Science group may have the opportunity to hire one researcher Senior: 0-3 yrs post PhD jobs.careers.microsoft.com/global/en/jo... Principal: 3+ yrs post PhD jobs.careers.microsoft.com/global/en/sh... Please note: our ability to hire this season is not certain

Anyone who shills Hydra gets a retweet. Not using it borders on malpractice.

If you're interested in the history of physics on a technical level, hit up Jorge with a follow and sub to his YouTube.

With all these all you need papers around I'm starting to think that people don't actually know what I need.

Now that @mkearnsphilly.bsky.social is here, this starter pack just became the real deal. Better refollow it. go.bsky.app/21nFz12

📦 Can frontier AI transform ANY physical object from ANY input modality into a high-quality digital twin that also MOVES? Excited to share our work,Articulate-Anything 🐵, exploring how VLMs can bridge the gap between the physical and digital worlds. Website: articulate-anything.github.io

Getting ready for #NeurIPS looking forward to meeting some old and new friends! Who is gonna be around?

Reskeeting this in case it got lost over Thanksgiving. Our lab is looking for a postdoc at the intersection of robotics and computer vision!

You just finished coding an experiment and now you run it. Running does not take days but also not seconds. Let's say it takes 25 minutes. What do you do in the time?

Been seeing a lot of RL architectures use layer norms as "design choice" without elaborating why it's needed. If you are curious take a look at our paper: arxiv.org/abs/2403.05996 TLDR: Layer norms stop your critic from diverging early on and stabilize training. cc @cvoelcker.bsky.social

Super excited about this new paper on human/model collaboration via agreement protocols! Joint work with the amazing @ncollina.bsky.social @vgup.bsky.social @aaroth.bsky.social 🌸

I have been saying "I made a blueskeet" and I have been told that that is not the term but maybe it should be? Or is there another accepted other term for skeeting?

Our lab is looking for a postdoc working on multi-robot autonomy, control and perception. This may include work on combining robots with recent advances in preception such as VLMs. If you are interested or are looking for more details about the project, hit up @ericeaton.bsky.social . #Robotics

Happy Food Coma Day to all who celebrate. 🦃🥧💤

It's almost time for #NeurIPS2024! I'll be there for workshops (and I'll be giving a talk at the Scalable Continual Learning workshop: sites.google.com/view/continu...). If you'll be around and want to chat, please reach out!