PSA: If your hands hurt from cutting spicy chillies, do not, i repeat, DO NOT, scratch your nose...

I’ve been thinking about a practical question and would love some opinions: How do your papers actually get discovered/cited? I was searching for recent work on high update ratio RL and found several very closely related papers tackling the same failure modes we study. None cited our earlier work.

🚀 Excited to share REPPO, a new on-policy RL agent! TL;DR: Replace PPO with REPPO for fewer hyperparameter headaches and more robust training. REPPO, led by @cvoelcker.bsky.social, will be presented at ICLR 2026. How does it work? 🧵👇

Should we use a target network in deep value-based RL?🤔 The answer has always been YES or NO, as there are pros and cons. @iclr-conf.bsky.social, I will present iS-QN, a method that lies in between this binary view, collecting the pros while reducing the cons🚀

Hyperparameters are a social construct (this is not irony or just sh*tposting)

OMFG my former boss got featured on r/LinkedInLunatics 😂😂😂

🎉 Really excited, our paper "XQC: Well-conditioned Optimization Accelerates Deep Reinforcement Learning" has been accepted at #ICLR2026. If you are interested in reinforcement learning, sample-efficiency, compute-efficiency go check it out. See you in Rio!

"PPO is not good, a thousand labs just reward tuned for it" is something I want to get tattooed so badly...

Looking at the math scores, I'd say at pass@1 (which is the thing RL actually optimizes for), the method is clearly outperformed by RL, at least within the math distribution. So the claim seems ... wrong? Am I missing something? Also, why do people write "RL doesn't work" papers so passionately?

This feels like true diffusion model training on LLMs 😂 start from pure information-free LinkedIn prose, iteratively refine Goethe and Shakespeare

Why is pass@k a metric? Does any proper LLM usecase actually generate 16 different answers and then picks the best one on ... vibes? This smells like massive cooking on the test set (or verifier reward).

I refuse to accept Texan weather as a real thing. It’s January for C sake! I want snow ❄️ and hot chocolate ☕️

I still hate coding with AI, but luckily, I hate writing data processing pipelines and webcrawlers more 😁

A new paper in Nature informs us there's a new AI benchmark called Humanity’s Last Exam. Yep, it's that same old HLE. They have submitted the paper 07 May 2025. And no, I don't know what the point of publishing it like that is either. Looks good on CVs, I guess.

❌ Institutional impact from cool new paper? 🦗 ✅ Institutional impact from making an LLM library installable on our hell-hole of a cluster? 🏆

How many great strategies for steering claude et al. are just sparkling placebo effect? Magical thinking? Asking for a friend...

Nothing has made me such a hardliner on rejecting unscientific woo in health/nutrition/etc as having a family member undergo tumour treatments over years. Last year alone HUGE breakthroughs in treatment of brain tumours has given countless patients a new lease on life www.nejm.org/doi/full/10....

How much people fetishizes absolutely terrible gig jobs like “checks notes” Uber driving and long-distance trucking in the comments is wild…

I need a list of what LLM RLVR modifications are ad-hoc hacks, and which one can be justified from principles... Seems like this is all over the place. For half of these modifications, a sane RL researcher says "duh", and for the other half it's "Ew, why???" ... I need to write this, right?

My job can be reliably done by a small script that std deviation is not actually any form of confidence interval for an estimator of the mean.

The other paper accepted to @iclr-conf.bsky.social 2026 🇧🇷. Our work on replicable RL sheds some light on how to consistently make decisions in RL. @ericeaton.bsky.social @mkearnsphilly.bsky.social @aaroth.bsky.social @sikatasengupta.bsky.social @optimistsinc.bsky.social

Or… you can chat with us in 🇧🇷 Rio 🇧🇷 as we are going to @iclr-conf.bsky.social to present our paper!!!

It is incredibly funny to get photos from my husband in Toronto of massive amounts of snow, while all of Austin is in apocalypse mode because there are 5 cm of fluff ❄️❄️❄️

OK, so, LLM coding models are kinda good, but LLM implementations themselves are ABSOLUTE DOGSHIT?! Like, wtf is the amount of breaking changes and random conflicts in absolutely every framework... This is worse than mujoco anno 2019

This week at reading group 📚 @pranav-nlp.bsky.social presented "You Cannot Sound Like GPT": Signs of language discrimination and resistance in computer science publishing. Paper: arxiv.org/abs/2505.08127 #NLProc

@icmlconf.bsky.social has cracked 25000 submissions 😂 (yes, there is a larger ICLR bulk in there, but still)

There are no in-the-box solutions to out-of-the box problems. You can tweak the scientific system, but without at least acknowledging the real extra-scientific pressures on the publication and review system, every conversation is incomplete. Publications support extra-scientific goals.

Nothing about publishing will improve until we collectively acknowledge that our systems were not built to be (a) hiring filters for generational big tech wealth and (b) the last remaining sane immigration paths to many countries, but espc. the US. Thanks for coming to my TED talk…

I need everybody to stop mentioning Sokol until we clean up after ourselves 😂

How the whole internet has kinda walked back phone browser compatibility is wild! I don’t have social media apps, because I never ever want to see notifications, and using browsers for any social is ROUGH

The UTexas TACC cluster is a master class in why you shouldn't just spend a bunch of money on a prestige project without talking to users...

hooking up Claude to my obsidian vault taught me that I have absolutely no clue for how to write good notes to myself 😅 it keeps asking questions about my notes and I’m like “I dunno, that probably made sense at some point??? Maybe???” Fun exercise, don’t know if my way of thinking is AI compatible

1️⃣ All of the paper we are currently pushing for upcoming conference deadlines are coming along nicely. 2️⃣ My colleagues are super supportive of my current post doc project. 3️⃣ I have finally made my favourite Makhani recipe reproducible.

🤔 Want to use REPPO (cvoelcker.de/projects/rep...) but hate jax? 🤔 😮 Want to have stable on-policy RL without filling your GPU with an enormous replay buffer? 😮 🤖 Are you a roboticist and just want your RL code to run? 🤖 🎉 Fear not, we started adding new REPPO versions! 🎉 github.com/cvoelcker/rs...

I've increasingly come to view cynicism as the seedbed of totalitarianism, both because it can paralyze the opposition and because it implicitly validates the might-makes-right ethic of the authoritarian.

Am I the only one who thought it reads kinda well? Washed up English major turned unhinged software dev is kinda my vibe tbh. I’d rather read this than most technical docs 😂

Dear Canadian mutuals: science doesn’t win when you try to spend money on poaching existing professors from the US. Science wins when you broaden the opportunities for juniors to become seniors. Sincerely, stop boasting about the money, start creating some junior faculty positions

Do add more MSG 😁

Job security, robust parental leave, health care, cost of living and espec. housing, good affordable child care, neighbourhoods where you don't have to drive the kids to EVERYTHING... There are a lot of progressive answers to these questions I feel??? Aside from the fact that the debate is weird...

The worst thing about modern AI tools is that I just got a transcript from a meeting, and apparently I ended almost every single sentence with either "right?" or "right!". I am verbalizing sentence endings all the time?! Please stop me!!!

ngmi

Fundamentally, one thing that’s fascinating is that income for major hyped tech companies is far more dependent on investment than on customers (would love some hard data on that). This completely changes incentive structures. The only ones out are NVIDIA, who simply created a de facto monopoly.

The more I claude-vibe, the less worried I am somehow about software devs as a profession. It is an amazing tool, but it shoots itself a lot if you don't guide it carefully, and without dev experience, I would have created a mess a lot of times today.

@eugenevinitsky.bsky.social can we have a gifted dogs research feed in your client, PLEASE!!!!

Every time I forget to bring a mask for a flight, I get sick after. I really, really, really should learn my lesson. Also, STOP FLYING SICK PEOPLE, I HEAR Y'ALL COUGHING!!!

With @icmlconf.bsky.social, RSS, and @rlca.bsky.social deadlines drawing near, to all of my fellow #rl and #robotics peeps I pose once again Gretchen's question: "Say, how do you stand towards baselines which were not published at Berkeley/Stanford?"

And this is a big part of why I left Frankfurt :D If Seattle is more sunny then your city, you gotta leave!

Honestly, @miterion (not on here sadly) dragging me (kicking and screaming) to learning Docker last year was probably the best thing that happened to me. Weird installation problems on server? F*** IT, CLAUDE, MAKE ME NEW DOCKERFILE!

Gentle reminder for the RL folks out there: RLC 2026 → Montreal 🇨🇦 (Aug 16–19) Abstract: Mar 1 | Submission: Mar 5 (AOE) | rl-conference.cc/callforpaper... Looking forward to receiving your submission!

One thing I don’t fully understand: why do students seem to feel way more comfortable asking ChatGPT to solve an assignment then to copy the solution from a friend? There seems to be some cultural difference.