Another exciting year for more RL! Submit your work to the RL conference and join us to talk a out how to make RL even better.

I'll be giving a talk tomorrow in the Embodied World Models for Decision Making workshop at #NeurIPS. I will present a number of recent works from my lab on combining foundational models with planning and RL. When: 11:30-noon Where: Level Room 30A-E

#VisionLanguage models are increasingly used for a wide range of problems, but seem complex to build. I wrote some code and recorded a tutorial in my lab yesterday to help others demystify how to create these models. #keepbuilding

My lab is looking for new students who are very passionate about foundational models and planning/RL/robotics. Apply via Mila. I will also be at #NeurIPS to discuss research ideas and opportunities. See notes below for application advice.

This not only seems illegal but is a breach of the social contract and trust in this research system. I can't see any good reason for breaking the confidentially of research grants and reference letters.

@pcastr.bsky.social is adding good clarity to the challenges of deep RL. Awesome stuff.

I have been seeing the subtleties in this race play out as destroying relationships. Families can fall apart because of the pressure to keep going in this race. Take care of yourselves everyone.

Does anyone in this universe know which human being accepts join requests for the #mlnews ml-news Google group? groups.google.com/g/ml-news. It says "This group is moderated and maintained by IMLS (www.machinelearning.org)." but that link just goes to the ICML webpage...

Surprise/empowerment/etc may be the fundamental objectives living organisms optimize, however it is very difficult to optimize these objectives. I will be giving a talk at international worlshop on #activeinference on how foundational models can help improve these methods.

For those interested in joining my lab, submit your application via the Mila form. This year I am particularly interested in students with skills/interests in robotics, reinforcement learning and, foundational models which will push forward the abilities of real world agents.

I am at #COLM2025 today to talk about AI, LLMs and simulation in the social simulation workshop. Come find me, happy to chat about all things AI, embodiment, and simulation.

The same could be said for science.

GRPO is more like REINFORCE than PPO. 1) It does not train a critic (no need with small variance) 2) The SCORE FUNCTION (difficult to call this an advantage) is over a batch using the same initial prompt (similar to the vine sample method from TRPO)

On my way to South Korea for a week packed with robotics at the conference on Robot Learning, Humanoids2025, and the global forum on mechanical engineering.

One of the most common logical fallacies I see is "GPUs are cooking, therefore progress." I see people with 1/10th the compute get 10x more progress because... they have a more thorough plan. #Moretimethinkinglesstimeburning

I suggest going out and talking to real people. They provide a much richer signal.

VLAs offer an avenue for generalist robot policies; however, naively following the action predictions leads to brittle or unsafe behaviours. We introduce VLAPS, which integrates model-based search with pre-trained VLA policies to improve performance without additional training.

Efficiency may be the most important. If we can't make these tools economical, they will not last.

My lab at @montrealrobotics.bsky.social was honoured to present our recent work to @mark-carney.bsky.social and Even Solomon explaining how AI enables new robotics that will drive innovation in Canada. It was a pleasure getting into the details with a quick dive into deterministic policy gradients!

Another fantastic Montreal Robotics Summer School! Thanks to our sponsors, organizers, and @mila-quebec.bsky.social, we doubled in size this year. Congratulations again to all the students who make this school happen, and for your progress in machine learning and robotics.

@rl-conference.bsky.social will be Montréal next year @umontreal-en.bsky.social!

After the LLM news using RL many are wondering whether progress in exploration or exploitation is needed to improve deep RL algorithms. This work introduces a new practical sub-optimality measure to understand how good an RL algorithm is at exploiting its experience.

Dessert at @rl-conference.bsky.social .

I have been cooking some code for training large generalist robotics policies that is almost ready for sharing! I will be presenting a tutorial on the code in a few weeks at an #IVADO LLM/VLM agents boot camp. Come checkout the most agentic system with full robotics control. ivado.ca/en/events/bo...

What is the best free service for compute? I am making some LLM and VLM fine-tuning examples in Python notebooks to share, but is there a free option with enough memory to load and train a model?

Who is interested in planning some socials at @rl-conference.bsky.social this year?

Being unable to scale #DeepRL to solve diverse, complex tasks with large distribution changes has been holding back the #RL community. In this work, we demonstrate that with the right architecture and optimization adjustments, agents can maintain plasticity for large networks.

How can we make behavioural cloning (BC) achieve better combinatorial generalization on out-of-distribution goals? We propose BYOL-γ: an auxiliary self-predictive loss to improve generalization for goal-conditioned BC. 🧵1/6

Great dialogue between Michael Littman and Kate Hartley to provide an overview of how RL, AGI and imitation learning have arrived where they are and the ingredients to make "AGI". @rldmdublin2025.bsky.social

In my last lecture on large-scale #robotlearning, I cover one of my most interesting directions, generalization across sequences and robots. This generalization across sequences of actions or states is a challenging, data-intensive process that requires experience across various robots and tasks.

Generalization across plans is a difficult, data-intensive process. Here, I cover learning representations to allow plans to generalize. I make connections between system identification, sim-to-real, meta reinforcement learning, and in-context learning and each method's limitations and generality.

Registered! See everyone in Ireland.

Tomorrow @iclr-conf.bsky.social we will present a method (SFM) for jointly learning state features and matching successor features, enabling strong imitation without action labels or adversarial training. Find us at Hall 3 + Hall 2B #572.

CoRL and @neuripsconf.bsky.social deadlines are approaching, and I have been thinking about how to help others improve their #writingstructure. In this note, I share how #technicalwriting is like writing code and how to organize writing to avoid **compile errors** in the reader's mind.

Autonomous learning agents need careful design and tooling to achieve useful levels of interaction in the real world. This lecture connects autonomous systems to recent ideas around #agenticmodels. These are key to learning from real-world interactions ( #reinforcementlearning ).

I am hiring another postdoc for my lab. Consider applying if you have #foundationalmodels, #robotics, or #reinforcementlearning skills. You will help create generalist real-world agents (robots) with a team of 20 working on these problems and overly competitive go-karting. Apply below.

I'm excited about today's talk at the @montrealrobotics.bsky.social #robotlearning seminar @mila-quebec.bsky.social ! We have @keerthanpg.bsky.social talking about her recent work at Google Robotics.

In general, #robots need to learn how to act while existing in a world with other agents. In this lecture, I cover the foundations of #multiagent #reinforcementlearning (MARL) and connect this to the recent work on reinforcement learning from human feedback (#RLHF).

Many recent #robotlearning works, such as Hi-robot, Gemini-Robotics, Helix, etc., use different types of hierarchical planning. In this lecture, I review hierarchical #reinforcementlearning for humanoid control, before discussing how these methods are used in recent works.

A few days ago, most of the @montrealrobotics.bsky.social lab went out for lunch at one of the local Cabanes à sucres! There was soooo much syrup.

We have levels of autonomy for #selfdriving cars. When will we have thought-out levels of autonomy for #humanoidrobots?

Providing goals as images to #robots is a flexible and accessible medium. In this lecture, I explain the best methods, challenges, and limitations of learning good #representations for images, starting with using more structure and moving on to using #largemodels and language as goals.

Profs or researchers who may be considering a move out of the US send me a message. #Canada has a long history of #fundamentalresearch and support for education. Multiple schools and organizations are working on making opportunities.

Recent #robotics #foundationalmodel papers use goal-conditioning and hierarchical planning. How to train these models? What goal distribution for training? What type of generalization will enable better reuse? I connect these concepts to foundational models for more depth..

How can control policies be trained from offline data and scaled to larger models and datasets? In this #robotlearning lecture, I cover common methods for training policies from fixed data, and then I discuss recent research on how to scale these methods to #largemodels and #largedatasets.

#DeepQlearning for continuous actions is key for controlling many types of #robots, but it has been tricky to train #largeModels to achieve those performance gains. In these lectures, I cover the fundamentals and explain how new research is bending the rules of the #deadlytriad to advance #scaling.

Needing #selfdriving, #reinforcementlearning, and #llm reviewers for #IROS2025 ! Please reach out.

Still more work to do but the demos from this model are very cool. spectrum.ieee.org/gemini-robot...

We are joined in our seminar tomorrow by Karl Pertsch a postdoctoral researcher at UC Berkeley, Stanford, and Physical Intelligence. His talk is titled "Unlocking Scalable Robot Learning in the Real World". See you at 1:30pm ET at www.youtube.com/@MontrealRob... #founationalmodels #robotics

#RobotLearning lecture update. These next lectures cover deep #qlearning fundamentals quickly and then get into the challenges of training #largemodels, maintaining the contraction property with target networks, network structure, and better optimizers.