Brazil shows how it’s done in a democracy

Excited to announce that our work on Reinforcement Learning for Arachnophobia treatment has been accepted at ACM Transactions on Interactive Intelligent Systems! We found that an RL agent could more effectively adapt VR spiders to achieve specified anxiety levels in users compared to current SOTA.

Was talking to a student who wasn't sure about why one would get a PhD. So I wrote up a list of reasons! www.eugenevinitsky.com/posts/reason...

Previous work has shown that programmatic policies—computer programs written in a domain-specific language—generalize to out-of-distribution problems more easily than neural policies. Is this really the case? 🧵

Previous work has shown that programmatic policies—computer programs written in a domain-specific language—generalize to out-of-distribution problems more easily than neural policies. Is this really the case? 🧵

If like me your Discover feed has been even worse lately and you are here for ML/AI news and discussion, check out these two feeds: - Paper Skygest - ML Feed: Trending Links below 👇

As AI agents face increasingly long and complex tasks, decomposing them into subtasks becomes increasingly appealing. But how do we discover such temporal structure? Hierarchical RL provides a natural formalism-yet many questions remain open. Here's our overview of the field🧵

As hot as this summer is, it’s also one of the coolest we’ll ever enjoy again. Just how much hotter and deadlier summers will get is still up to us. Right now we’re working hard to make them worse 🎁 link to my @opinion.bloomberg.com column: www.bloomberg.com/opinion/arti...

Hiring a postdoc to scale up and deploy RL-based planning onto some self-driving cars! We'll be building on arxiv.org/abs/2502.03349 and learn what the limits and challenges of RL planning are. Shoot me a message if interested and help spread the word please! Full posting to come in a bit.

We’re extending the AIIDE deadline! Partially due to author requests, partially due to a significant increase in submissions meaning I need to increase the PC!

🧵1/ New paper! 📄 Subgoal-Guided Policy Heuristic Search with Learned Subgoals, led by my PhD student @tuero.ca. arxiv.org/pdf/2506.07255 This paper follows the Levin tree search (LTS) research line and focuses on learning subgoal-based policies.

I enjoyed reading this and the comments within it.

After literal years of being down due to security issues, my blog is now back up, including my one post on "How to Write an AIIDE Paper" www.guzdial.com/blog/how-to-...

We are very happy to report that the second edition of our textbook on Artificial Intelligence and Games is now finally published! This book is a thorough update of our popular textbook, trying to provide a comprehensive coverage of the many aspects of and use cases for AI in games.

Excited to announce that the second edition of the PCGML textbook by @sampsnodgrass.bsky.social, @autumnsburg.bsky.social, and myself is up! This version is a major update of the first, with a new chapter on GenAI and updates on the last two years of research. link.springer.com/book/10.1007...

We have extended the submission deadlines for these workshops. Programmatic Representations for Agent Learning (ICML) - Vancouver. - Deadline: May 30, 2025 (pral-workshop.github.io) Programmatic Reinforcement Learning (RLC) - Edmonton. - Deadline: June 6, 2025 (prl-workshop.github.io)

📢 I'm very excited to release AgarCL, a new evaluation platform for research in continual reinforcement learning‼️ Repo: github.com/machado-rese... Website: agarcl.github.io Preprint: arxiv.org/abs/2505.18347 Details below 👇

🧵1/ New paper! 📄 InnateCoder: Learning Programmatic Options with Foundation Models This is Rubens Moraes' final chapter of his PhD thesis from Universidade Federal de Viçosa, Brazil, in collaboration with Quazi Sadmine and Hendrik Baier. arXiv: arxiv.org/abs/2505.12508

📢 I'm happy to share the preprint: _Reward-Aware Proto-Representations in Reinforcement Learning_ ‼️ My PhD student, Hon Tik Tse, led this work, and my MSc student, Siddarth Chandrasekar, assisted us. arxiv.org/abs/2505.16217 Basically, it's the SR with rewards. See below 👇

🧵1/ New paper! 📄 InnateCoder: Learning Programmatic Options with Foundation Models This is Rubens Moraes' final chapter of his PhD thesis from Universidade Federal de Viçosa, Brazil, in collaboration with Quazi Sadmine and Hendrik Baier. arXiv: arxiv.org/abs/2505.12508

🎉 Super excited: today @sharky6000.bsky.social is presenting our new algorithm Progressive Hiding at #AAMAS2025! It's a learning method for games with imperfect information. 🔗 See his post: bsky.app/profile/shar... Wish I could be there! 😢 1/6

Are you interested in programmatic representations in machine learning? Please consider submitting your work to these workshops: one at ICML and another at RLC. 🧵

Wow! Great list of papers on the computational capability of neural networks.

🎓 PhD position available! Join our interdisciplinary research project on causal agent-based modelling! 🔍 Looking for curious minds with a MSc degree (or near to completing one) in CS/AI/related fields. 📍 Location: Utrecht University, NL 🗓️ Deadline: 16 June 2025 📩 Info: www.uu.nl/en/organisat...

I enjoyed reading this thread.

A bit of an old paper but I'm still excited about it, I've never posted about it, and we just released code for the main algorithm: Population RL (PopRL). 👇 Population-based Evaluation in Repeated Rock-Paper-Scissors as a Benchmark for Multiagent Reinforcement Learning, accepted (TMLR '23) 🧵 1/N

So happy to see that ICLR will go to Brazil next year!

rlhfbook also available on arxiv for SEO 😀 happy friday arxiv.org/abs/2504.12501

We have a new open position for a PhD student to work on Reinforcement Learning for Cyber Security! Check out vacancies.maastrichtuniversity.nl/job/Maastric... for details, and feel free to contact me for any info as well. Application deadline: May 11 Reposts much appreciated.

Three postdoc positions in HCI & Videogame Theory at the IT University of Copenhagen. Deadline May 15. "Candidates will investigate current theory use in HCI/games research and practice, develop approaches to make theory more usable, and evaluate the effectiveness of such interventions"

👾Call for Papers for #AIIDE25 is live! 👾 Join us in Edmonton, Canada where we will be celebrating over 20 years of AIIDE with a special theme: Strong Foundations! We encourage the submission of work that builds on prior work from AIIDE proceedings. 🔍Info in replies!

We are very pleased to announce that AIIDE'25 will be back at the University of Alberta in Edmonton, Alberta, Canada from November 10-14! Paper deadline June 21 AOE. Join us with your work at the intersection of AI and Digital Entertainment! sites.google.com/ualberta.ca/...

The Canadian AI Safety Institute (CAISI) Research Program at CIFAR is now accepting Expressions of Interest for Solution Networks in AI Safety under two themes: * Mitigating the Safety Risks of Synthetic Content * AI Safety in the Global South. cifar.ca/ai/ai-and-so...

There is a postdoctoral position for research on adversarial robustness at UBC, supervised by Mathias Lecuyer (@mathias-lecuyer.bsky.social), Geoff Pleiss, and Nidhi Hegde. Please spread the word! 🇨🇦 docs.google.com/forms/d/e/1F...

This recognition is long overdue. I was so happy to wake up to this news today. awards.acm.org/about/2024-t...

On why it’s time to remove the bias-variance tradeoff from machine learning 101.

This is one of my favorite types of papers, where strong empirical evidence points in a direction different from where most people had been looking.

Model-free deep RL algorithms like NFSP, PSRO, ESCHER, & R-NaD are tailor-made for games with hidden information (e.g. poker). We performed the largest-ever comparison of these algorithms. We find that they do not outperform generic policy gradient methods, such as PPO. arxiv.org/abs/2502.08938 1/N

We've built a simulated driving agent that we trained on 1.6 billion km of driving with no human data. It is SOTA on every planning benchmark we tried. In self-play, it goes 20 years between collisions.

Martin led this great work, check it out. For a dinosaur like me, let me say that, in more classical RL terms, this is a demonstration of how we can effectively combine options and LLMs through programmatic policies.

Can AI agents adapt zero-shot, to complex multi-step language instructions in open-ended environments? We present MaestroMotif, a method for skill design that produces highly capable and steerable hierarchical agents. Paper: arxiv.org/abs/2412.08542 Code: github.com/mklissa/maestromotif

I am re-reading "Learning Differentiable Programs with Admissible Neural Heuristics." This is still one of my favorite papers from the past 5–6 years. arxiv.org/pdf/2007.12101

RLC call for workshops is out rl-conference.cc/callforworks...! Submissions open on Feb 3rd, deadline on March 7, with the workshops on Aug. 5th. Last year's workshops were inimitable and we @claireve.bsky.social and Josiah Hanna) look forward to your amazing proposals

We just released extensive instructions for all the reviewing roles at @rl-conference.bsky.social; ranging from SAC to TR. We are trying something different here that we believe can be better. To ensure we are open about it, we made those instructions public: rl-conference.cc/reviewinstru...

Very proud to share that Jialiang Yan from University of Alberta, who I've had the privilege to work with, has been recognized with an honourable mention for CRAs outstanding undergraduate research award!!! www.ualberta.ca/en/computing...

Day 1 of our workshop on Private Data Science is underway in sunny San Diego thanks to the encore institute. Co-organized with Rachel Cummings, Amartya Sanyal @amartyasanyal.bsky.social, Clément Canonne @ccanonne.bsky.social and myself, itll be a great three days on privacy! encoredp.github.io

Brandon is a wonderful research colleague and I could not endorse enough trying to work with Brandon

Three of my (stellar) students are looking for research-oriented internships in PL/FM (but also LLM for code) this coming summer: - Jinwoo Kim (cseweb.ucsd.edu/~jik083/) - Shaan Nagy (cseweb.ucsd.edu/~shnagy/) - Xuanyu Peng (home.dofy.top)

This is where programmatic representations can help. In supervised learning, we already have inspiring examples like FlashFill, where a strong inductive bias through a highly constrained hypothesis space allows one to learn complex hypotheses from only 2-3 examples.