🚀 PhD position in #NeuroAI & neurodevelopment 🚀 Co-supervised by Sarah Lippé and myself, to investigate visual processing & cognition abnormalities in children with neurodevelopmental disorders in a neuroAI framework. Full project details and how to apply here: tinyurl.com/kbuyntpn 🧠🤖 📈

If you have a Reinforcement Learning paper accepted at TMLR, JMLR, JAIR, AIJ, or MLJ, you can use this wonderful opportunity to present your work and meet your peers at RLC in Montreal, Canada. 🇨🇦

📣 There's never a "best" time to share important updates, especially after sitting on this for so long. I'm joining the faculty @brighamyoungu.bsky.social this Summer as an Assistant Professor in the CS Dept, in preparation for the coming school year. Lots of excitement and a fair bit of nerves. 🧵

Do you use often use PPO, but wish you could use something just better? Try REPPO: Relative Entropy Pathwise Policy Optimization! Project Page: cvoelcker.de/projects/rep...

At #ICLR2026 presenting our first poster in the morning on Relative Entropy Pathwise Policy Optimization. Stop by at #4613. 🧑‍🎓 @cvoelcker.bsky.social, @axelbrunnbauer.bsky.social, Michal Naumann, Pieter Abbeel, @ericeaton.bsky.social, Radu Grosu, @sologen.bsky.social @igilitschenski.bsky.social

Hypothesis: People have been gradually shifting to write more like ChatGPT and alike. They use structures such as "This is not only X; but it is also Y". These struct. are natural part of lang, but either 1. they're becoming more prevalent, 2. I've become more sensitive to them.

We have a new PhD Candidate in town: @tylerkastner.bsky.social Looking forward to all the new work you will be doing on Distributional Reinforcement Learning.

We have the keynote speakers for RLC2026 now! Thrilled to welcome Rika Antonova, Sheila McIlraith, Marc G. Bellemare, Danijar Hafner, Balaraman Ravindran! Details: rl-conference.cc/index.html The RL community is coming together this August in Montréal, Québec, Canada. Hope you make it!

Palantir has student data, including immigration status, from the ed tech discussion platform Piazza. Palantir paid Piazza $916,000 for access to this data. www.sec.gov/Archives/edg... I blew the whistle on this in 2016 and the CEO contacted my employer.

Happy Norooz, the Persian new year 1405/2585, the equinox, and the beginning of spring!

Following advice by the always-wise @eugenevinitsky.bsky.social , I am trying to get back into the habit of blogging (again) ✏️! Featuring today's post: How to pick an RL algorithm for your problem cvoelcker.de/blog/2026/ch... Please share and give feedback! #reinforcementlearning

In light of the ongoing conflict in the Middle East, RLC decided to remove the abstract deadline: rl-conference.cc/callforpaper... The only deadline is for the full paper: Mar 5(AOE) openreview.net/group?id=rl-... Affected folks may also contact the PCs to discuss deadline extensions before Mar 5.

Ali Khamenei is in hell. The world is a better place now!

RLC 2026 Call for Workshop is live on OpenReview! Submission deadline: Mar 12 (AoE). Full details here: rl-conference.cc/call_for_wor... @glenberseth.bsky.social @eugenevinitsky.bsky.social @twkillian.bsky.social @schaul.bsky.social @sologen.bsky.social @audurand.bsky.social @bradknox.bsky.social

Submit your RL papers to RLC! This is now perhaps the best venue for RL researchers.

I am rerunning my class on robot learning this year, and I plan to push many code examples to help others get to the ugly details fast. One of these details is how BC gets off track as network sizes change. Blog and notebook below.

🚀 Excited to share REPPO, a new on-policy RL agent! TL;DR: Replace PPO with REPPO for fewer hyperparameter headaches and more robust training. REPPO, led by @cvoelcker.bsky.social, will be presented at ICLR 2026. How does it work? 🧵👇

The compliment of the day: "What’s unusual is your willingness to follow the logic all the way through instead of stopping where it becomes socially awkward".

Has taken a long time to polish, but slowly becoming very proud of rlhfbook.com and do think it's a great resource for many people. A lot of hours (and tokens and reader feedback) going into making it right.

Their silence is deafening.

A significant hurdle of the empirical RL and the broader AI research is caused by the limitations of the environments in which our agents learn and build their "artificial minds". This should be compared with the richness of the real-world in which a human child flourishes.

Grading ...

Hi RL Enthusiasts! RLC is coming to Montreal, Quebec, in the summer: Aug 16–19, 2026! Call for Papers is up now: Abstract: Mar 1 (AOE) Submission: Mar 5 (AOE) Excited to see what you’ve been up to - Submit your best work! rl-conference.cc/callforpaper... Please share widely!

I came across this interview of Edward Witten (@edwardwitten.bsky.social) by Brian Greene. I don't usually watch long YT videos, but this one is quite curious and "electrifying". I recommend it if you're interested in the state of String Theory. www.youtube.com/watch?v=sAbP...

Ah ... you have to be an American to become rich in Canada!

What are you favourite Imitation Learning and Inverse Reinforcement Learning papers? What are the essential papers? It doesn't matter much whether they are new or old, but I prefer a conceptually elegant and mathematically solid work.

"No agent lives in a vacuum; it must interact with other agents to achieve its goals. Reinforcement learning is a promising technique for creating agents that co-exist, but the mathematical framework that justifies it is inappropriate for multi-agent environments.

Observation: Using ChatGPT to do homework assignments shows off when you do the Final Exam without ChatGPT!

J. M. Coetzee: And for whom, anyway, do we do the things that lead to Nobel Prizes if not for our mothers? "Mommy, Mommy, I won a prize!" "That’s wonderful, my dear. Now eat your carrots before they get cold." www.nobelprize.org/prizes/liter...

6 décembre 2025 – 36 ans après – Souvenir et recueillement 1/2 Nous continuons à nous souvenir des treize étudiantes et d'une membre du personnel de Polytechnique ayant perdu la vie le 6 décembre 1989 et des personnes qui en sont restées meurtries. polymtl.ca/6decembre

🥁 Announcing the 2025 TMLR Outstanding Certification! The committee decided upon 1 recipient and 3 finalists. Read on to learn more! Committee: Pablo Samuel Castro, Pin-Yu Chen, Vincent Dumoulin, Amir massoud Farahmand, Andreas Kirsch, Jasper Lee, Jeffrey Pennington, Colin Raffel, Chang Xu 1/n

If you are into Wittgenstein with a dose of QM, this might be interesting: Tractatus Quanticum [Spoiler] "What we do not have information about, we must pass over in silence." philsci-archive.pitt.edu/27289/1/main... P.S: I haven't read it closely.

I agree! My presentations have real photos (either mine or credited appropriately), simple illustrations, and a watercolour-painted robot, made by a real artist @fvreilly.bsky.social .

I'm looking for graduate students to join my group in fall 2026. We work at the intersection of Computer Vision, Deep Learning, and Robotics. The goal of our work is to create and understand organic 🍀 simulation systems, i.e., controllable data-curation engines built from real-world data. 1/n

This 1978 short video from Arthur C. Clarke is interesting. He talks about computers that learn, the possibility of them outpacing their creators (aka singularity), and touches upon the job market and the the purpose of life in light of that possibility. www.youtube.com/watch?v=I9DF...

A quarter-baked thought: What is the Aha! Distance between two related major theories/ideas? I mean something like this: Within the Kuhnian normal science, TO (Theory Old) is believed until we have a scientific revolution when we start seeing things through TN (Theory New).

Can’t agree more. Still trying to wrap my head around the budget news. I want to move on, but honestly can’t. Yes, there’s some self-interest here, but adding another well-established researcher to an already deprived funding landscape isn’t exactly encouraging.

The 🇨🇦 Standing Committee on Science and Research passed a modified resolution this afternoon that the requested ricouncil applicant and reviewer data can be aggregated and anonymized. Thanks to the many researchers who signed letters pointing out legitimate data privacy concerns. 👏👏

It’s hard not to feel insulted by the Canadian federal budget’s implication that researchers currently working in Canada aren’t good enough, so they’ll spend a bunch of taxpayer money to bring in “top talent” from elsewhere.

Canadian researchers should be aware the there is a motion before the Parliamentary Standing Committee on Science and Research to force Tricouncils to hand over disaggregated peer review data on all applications: Applicant names, profiles, demographics Reviewers names, profiles, comments, and scores

As Transactions on Machine Learning Research (TMLR) grows in number of submissions, we are looking for more reviewers and action editors. Please sign up! Only one paper to review at a time and <= 6 per year, reviewers report greater satisfaction than reviewing for conferences!

Happy Diwali!

Apply to join one of the top AI research 🤖 institutes in the world in the wonderful city of Montréal! I will be recruiting 1-2 students this year. Just mention my name in your application.

Mirrorless Mirror Descent: A Natural Derivation of Mirror Descent (Gunasekar, Woodworth, Srebro at AISTATS, 2021) They show that the Mirror Descent algorithm is a particular way of discretization a certain geometry-aware gradient flow. proceedings.mlr.press/v130/gunasek... Interesting paper!

cvoelcker.de/blog/2025/re... I finally gave in and made a nice blog post about my most recent paper. This was a surprising amount of work, so please be nice and go read it!

Understanding the Effect of Stochasticity in Policy Optimization (NeurIPS 2021) by Jincheng Mei, Bo Dai, Chenjun Xiao, @skiandsolve.bsky.social, Dale Schuurmans. Interesting paper on Policy Gradient (PG) methods! PG >> NPG or PG << NPG?! It depends on your estimator. arxiv.org/abs/2110.15572

The philosopher John Searle died recently. He was one of the famous contemporary philosophers about whom I kept reading or hearing, mostly because of his Chinese Room argument. Reading some comments by those who have actually met him, it seems that he was a character!

Shanah tovah! שנה טובה

Three years ago today, #MahsaAmini was murdered by the Islamic Republic. Her death started the hopeful Women, Life, Freedom movement in Iran and across the globe. Mahsa, Nika, Sarina, and 100s of others are not among us anymore, but their influence has changed Iran forever.

The citation depth of ML is shallow: most people don't meaningfully cite a more than a few years old paper (yes, textbooks and obligatory classics aside). Why? Many reasons, including that the authors probably haven't actually read the old papers. Why? Two reasons: