Three years ago today, #MahsaAmini was murdered by the Islamic Republic. Her death started the hopeful Women, Life, Freedom movement in Iran and across the globe. Mahsa, Nika, Sarina, and 100s of others are not among us anymore, but their influence has changed Iran forever.

The citation depth of ML is shallow: most people don't meaningfully cite a more than a few years old paper (yes, textbooks and obligatory classics aside). Why? Many reasons, including that the authors probably haven't actually read the old papers. Why? Two reasons:

I recently learned the existence of something called Descriptive CV. This is a type of CV that must adapt to the grant application, so whenever you write a research proposal, you have to write a new CV too. Wonderful!

Big news, everyone!

It all started with a proposal, 60 years ago! "A Proposal for Dartmouth Summer Research Project on Artificial Intelligence". jmc.stanford.edu/articles/dar...

Freaking Famine! The situation in Gaza will be the stain on our generation and all whose support led to this.

The Prime Minister is primed to the Policy Search methods. The AI minister, however, is more into the Critique.

I finally finished it! www.victorzhong.com/writing/refl...

What are we talking about when we talk about Dynamic Programming? #ReinforcementLearning

Skimming the timeline, I misread a tweet as "Planning as Syllabus". That made me pause. It sounds like an interesting ideas, doesn't it? What does it exactly mean though? Not sure, but maybe we can think of planning of the future as designing a syllabus for our learning agent.

From an obituary: "Finally, J. would undoubtedly instruct all who read this to pay your taxes happily, fight like hell to protect universal health care, call your mother, squeeze your babies (fur ones included) and above all, be a good citizen."

Let me introduce Dr. Claas Voelcker! @cvoelcker.bsky.social Claas defended his PhD on Friday (Learning to model what matters–Representations and world models for efficient reinforcement learning) at the University of Toronto. Claas is my 3rd graduated PhD (🌟🌟🌟), and I'm so proud of him!

Finally, it is done! You may now all call me Dr. Claas (and then immediately laugh at me for being pretentious enough to use the title). I am super happy/relieved/exhausted to announce that I passed my thesis defense yesterday! #PhDone #mlsky

🥳Let me introduce you Dr. Avery Ma! 🥳 Avery defended his PhD dissertation yesterday (Understanding Adversarial Robustness in Deep Learning) at the Department of Computer Science @utoronto.ca. Avery is my 2nd graduated PhD, and I am so proud of him! averyma.github.io

While folks are discussing how well LLMs are solving IMO problems (however impressive, still sub-Bronze), I am hoping to see some of these brilliant students in my application pool in 4 years!

🔥 Presenting Relative Entropy Pathwise Policy Optimization #REPPO 🔥 Off-policy #RL (eg #TD3) trains by differentiating a critic, while on-policy #RL (eg #PPO) uses Monte-Carlo gradients. But is that necessary? Turns out: No! We show how to get critic gradients on-policy. arxiv.org/abs/2507.11019

If you are a part of the ICML community (authors, reviewers, etc. in the past few years): Don't forget to vote for the Board of Directors. The deadline is today (July 16).

I will not be at #ICML2025 this year, but 3 of my PhD students at 🤖 Adage (Adaptive Agents Lab) 🤖 are, presenting 3 papers. ⭐ Avery Ma ⭐ Claas Voelcker (cvoelcker.bsky.social) ⭐ Tyler Kastner Meet them to talk about Model-based RL, Distributional RL, and Jailbreaking LLMs.

In 🇨🇦 we also use a note that if a local product isn't available, what non-🇺🇸 substitute is acceptable: 🍈 from 🇲🇽 or 🇬🇹? 🍯 from 🇦🇷 or 🇿🇦?

First position paper I ever wrote. "Beyond Statistical Learning: Exact Learning Is Essential for General Intelligence" arxiv.org/abs/2506.23908 Background: I'd like LLMs to help me do math, but statistical learning seems inadequate to make this happen. What do you all think?

What do we talk about when we talk about the Bellman Optimality Equation? If we think carefully, we are (implicitly) making three claims. #FoundationsOfReinforcementLearning #sneakpeek

This could be a plot for a sci-fi story, but not anymore: "AI authors hacks an AI reviewers to accept their papers in a conference for humans." (Probably not exactly the case here, but not far from it.)

🇨🇦 Happy Canada Day! 🇨🇦 I'm grateful about many good things about Canada. But as I often write on my students' papers and dissertation drafts, "Good job! There is room for improvement!". Let's do it!

Thanks to the series Love Death + Robots (highly recomm), I recently discovered @michaelswanwick.bsky.social. One of his short stories, The Dead (1996), reminded me of concerns about the effect of AI on economy and relevance of humans, even though it's about zombies! Read: reactormag.com/the-dead/

proud to share a survey of state representation learning in RL that my student ayoub echchahed and i prepared, that was just published on @tmlrorg.bsky.social ! this was the bulk of ayoub's masters thesis and he put a lot of work and care into it! a few details in thread below... 1/

Hiring a postdoc to scale up and deploy RL-based planning onto some self-driving cars! We'll be building on arxiv.org/abs/2502.03349 and learn what the limits and challenges of RL planning are. Shoot me a message if interested and help spread the word please! Full posting to come in a bit.

Give @pedanana.bsky.social a follow everybody (and put her on some RL lists!) She was our longest running undergrad intern, showed incredible dedication on the hardest and most frustrating projects, and is now starting her grad school career with the amazing Martha White (who is not on Bluesky?!)

Works that use #VAML/ #MuZero losses often use deterministic models. But if we want to use stochastic models to measure uncertainty or because we want to leverage current SOTA models such as #transformers and #diffusion, we need to take care! Naively translating the loss functions leads to mistakes!

For more details, feel free to come chat with us in Vancouver⛰️🌲🌊 and check out our paper🤖! www.arxiv.org/abs/2505.22772

Would you be surprised to learn that many empirical implementations of value-aware model learning (VAML) algos, including MuZero, lead to incorrect model & value functions when training stochastic models 🤕? In our new @icmlconf.bsky.social 2025 paper, we show why this happens and how to fix it 🦾!

To pivot or not to pivot your research topic?! This paper shows the existence of a pivot penalty: "the impact of new research steeply declines the further a researcher moves from their previous work". www.nature.com/articles/s41...

“It’s not ‘complicated.’ “ And it’s too late. But let the image of Ward grab hold and save what lives we still can.” A short column for @thestar.com. #Gaza

📢 I'm very excited to release AgarCL, a new evaluation platform for research in continual reinforcement learning‼️ Repo: github.com/machado-rese... Website: agarcl.github.io Preprint: arxiv.org/abs/2505.18347 Details below 👇

When you are updating your Canadian Common CV, you have a choice: consider it as a task to be done, or a meditative exercise. Your choice will decide your mental health.

Are you an RL PhD at Harvard who has had your funding wrecked by the government and working on topics related to multi-agent? Reach out, I am happy to try to find a way to support you.

Llama 2 on Commodore C64! I have not played with it, but this looks impressive! (cheating: it requires 2MB, and not 64KB.) A good test of algorithmic progress is to show the method works well and is efficient with an old computer hardware. My favourite? Pentium era! github.com/ytmytm/llama...

Sometimes while reading a paper or thinking about a topic I wish I knew how researcher A (expert in topic X) would react to the results by researcher B (expert in topic Y), as I suspect a close connection/relation between X and Y, but apparently, AFAIK, it is not realized yet.

I didn't expect to read about the benefit of over-parametrization in avoiding poor local minima in one of earliest work on backpropagation: "Learning representations by back-propagating errors" (1986) (by Rumelhart, Hinton, and Williams): iro.umontreal.ca/~vincentp/if... (1/4)

This week, Mila researchers will present 15 papers at AISTATS 2025 in Mai Khao, Thailand. Every day, we will share a schedule featuring Mila-affiliated presentations. Day 1 #AISTATS mila.quebec/en/news/foll...

Add "Cs" = "ch" and you are all good! @skiandsolve.bsky.social

Many great papers from Mila! Two by my team at the Adaptive Agents Lab (Adage) together with collaborators: A Truncated Newton Method for Optimal Transport openreview.net/forum?id=gWr... MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL openreview.net/forum?id=6Rt... #ICLR2025

I'll be at @iclr-conf.bsky.social next week! Ping me if you want to chat about RL, life, or the universe! I'll be presenting our MAD-TD paper openreview.net/forum?id=6Rt... at the poster session, and I will also most likely be lurking at the Robot Learning workshop.

The Conservative Party of Canada plans to "Reduce Funding for Artificial Intelligence Initiatives" by $2.275B between 2025-2029. Conclude however you want! (You can find it on page 27 of their platform: canada-first-for-a-change.s3.us-west-2.amazonaws.com/20250418_CPC...) #cdnpoli

rlhfbook also available on arxiv for SEO 😀 happy friday arxiv.org/abs/2504.12501

Widsom #53: A sign of a good research collaborator (peer, student, or supervisor) is that you can say possibility stupid things and none of you feel bad about it. The cost of exploration is zero. Wisdom #53.a: This applies more broadly than collaborators.

Need an editor? I've got years of experience and reasonable rates! Student discounts available. www.michelledesveaux.com

Fix the brain drain first, then attract new talents! Attracting talent from outside Canada is good in principle, but not at the price of forgetting about all the brilliant researchers who are already in Canada looking for a good position within Canada. 🇨🇦

Awkward: Regularly introducing yourself to the same person over the years. No sign of recognition from the past. No sign of dementia. Positive sign of being perceived as a hotshot.

I’ve come to realize that many survey papers are bad! What I want from a survey is to give me a reasonable understanding of a research area + pointers to important classical/foundational papers + a taste of recent advances.

Of course, the Fisher Information Matrix of softmax model is singular! Precise: given a vector u, softmax(u) has a singular FIM. Intuition: Adding a const c to u leads to the same distribution. FIM being the metric of the prob manifold should show a change, unless singular.