Every LLM eval uses Bradley-Terry Elo rankings. Almost none report uncertainty. Should we trust them? Maybe there is something better... 👇 (1/5)

Pokemon is truly the pareto frontier of agent research - The RPG requires an autonomous embodied agentic agent with perception, planning, memory, and control - VGC and Gen 9 OU penalize erroneous actions with fast-paced opponent-modeling in short games (1/3)

Trying to get a post ready but bluesky won’t let me post on desktop!!! If you want users here you need a user experience!!!

You probably aren’t reading enough papers. You probably didn’t cite the 10 closest papers to your work Thus, LLMs probably have a better understanding of where your paper sits in the literature ¯\_(ツ)_/¯

The most interesting papers arent being published at the “prestigious” venues anymore. Where are you publishing and what do you work on?

🚨 Hackathon Weekend! 🚨 Jumpstart your PokéAgent Challenge submission ahead of NeurIPS! 📅 Sept 13–14 ✅ Leaderboards reset Sat 10AM EDT 🎙️ Lightning talks in LLMs, RL, and Pokemon 💬 Live Office hours 🏆 $2k in prizes

The NeurIPS 2025 PokéAgent Challenge is offering compute credits, courtesy of our sponsor Google DeepMind, to help you train bigger models & run more experiments. 📌 To apply: 1️⃣ Make a submission to Track 1 or 2 at pokeagent.github.io 2️⃣ Fill out the compute credit form on the site

Mad about data centers? Call your reps to build more nuclear

Hey #academics Why are neurips workshop deadlines due a month before main track acceptances? Seems counterintuitive to have the two tracks compete with each other #machinelearning

The NeurIPS 2025 PokéAgent Challenge is offering compute credits, courtesy of our sponsor Google DeepMind, to help you train bigger models & run more experiments. 📌 To apply: 1️⃣ Make a submission to Track 1 or 2 at pokeagent.github.io 2️⃣ Fill out the compute credit form on the site

If your final product doesnt reason in-context, how is it supposed to meta-learn and address distribution shifts and environment changes?

Papers are dead. Maybe it is time to start the youtube channel…

Viral paper out today about predicting brain stimulus from video inputs. as always dont overfit on first order responses. If you oversaturate stimulus, people will stop using the product(people uninstalling IG because it is too addicting) The attention economy must be modeled as a multi-agent system

🚀 New preprint! 🤔 Can one agent “nudge” a synthetic civilization of Census‑grounded agents toward higher social welfare—all by optimizing utilities in‑context? Meet the LLM Economist ↓

🚀 New preprint! 🤔 Can one agent “nudge” a synthetic civilization of Census‑grounded agents toward higher social welfare—all by optimizing utilities in‑context? Meet the LLM Economist ↓

🚀 Launch day! The NeurIPS 2025 PokéAgent Challenge is live. @neuripsconf.bsky.social Two tracks: ① Showdown Battling – imperfect-info, turn-based strategy ② Pokemon Emerald Speedrunning – long horizon RPG planning 5 M labeled replays • starter kit • baselines. Bring your LLM, RL, or hybrid agent!

🚀 5 days until my ICML spotlight poster! Key insights we’ll unpack: • Base LLM + test-time planning • Game-theoretic scaffolding • Context-engineered opponent prediction • Comparative LLM-as-judge (relative > absolute) Catch me Thu Jul 17, 4:30-7 PM PT👇

Heading to #ICML2025 next week! If you’re into all things API (Artificial Pokémon Intelligence) from our PokéChamp spotlight to the upcoming NeurIPS PokeAgent Challenge, LLM-agent scaffolding & reasoning, or mechanism-design nudging, let’s connect. DMs open!

Also the Pokemon Agent challenge by @sethkarten.ai @stephmilani.bsky.social and others! pokeagent.github.io

Social media takeoff is hard. Bluesky still lacks the capability to compete with twitter

Excited to announce that I will be spending the summer at @Waymo on the simulation realism team! I’ll be working on learning to generate simulated worlds. 🚙🚙🚙 Send me a message if youre in the bay area and want to chat!

Excited to share that the PokeAgent challenge was accepted as a NeurIPS competition! This should serve as an excellent benchmark for competitive games AND ‘speedrunning’ the RPG. I hope to see both the RL and LLM agent communities working together here to eval agents in Pokemon More info soon👀

Scaffolding is a really bad term. It is amazing how we have gone so end-to-end that we cant imagine when the model was just one component in an architecture

Researchers need to stop working on low-hanging fruit. Leave that for the engineers. Your job is difficult answering questions that people will push back on

Wow, this is officially an ICML spotlight! See you in Vancouver :)

using bluesky doesnt mean one has to be anti-AI 👍👍

Ive been working a lot on mechanism design with LLMs recently so I am deeply interested in seeing this done correctly. My take on this is that being able to elicit change isnt enough for this type of work. It also matters what the end goal is.

What happens to TRI though? I thought they had an AV division. Also Toyotas arent EVs so I am confused how the driving tech stack would work. I think Waymo is just diversifying their risk into personal vehicles

Insane new study from zurich studies the influence of the LLMs for persuasion on the r/ChangeMyView subreddit. Let's just say people are outraged... Is the study justified since bots are already rampant on reddit? Or does this cross ethical lines?

Some of the anti-AI stuff feels a bit like when people would say "don't use Wikipedia as a source." It's just like anything else, a piece of information that you weigh against multiple sources and your own understanding of its likely failure modes

IJCAI workshops look so good this year. I really feel IJCAI/AAAI has some pretty amazing and overlooked work in the ICLR/ICML/NeurIPS era. Maybe we need a AAAI/RLC/NeurIPS era

The fact that there are two EV tiny trucks company launches this week is pretty cool to us kei truck fans

Incredibly bullish on openai

In my academic arbitrage era (reading papers from the 70s and 80s from other fields and saying but what if it were an LLM)

🥥🥥 Announcing the Second (!!) Coordination and Cooperation for Multi-Agent Reinforcement Learning (CoCoMARL) Workshop at RLC 2025 @rl-conference.bsky.social 🥥🥥 Please see our website for more information and the call for papers! sites.google.com/view/cocomar...

I'm excited to release the PokéChamp dataset! 🎮 2 million cleaned battle logs from Pokémon Showdown to help train expert-level AI agents for competitive Pokémon. Check out the data on huggingface and code on GitHub! 🔗 huggingface.co/datasets/mil... 🔗 github.com/sethkarten/p...

Wow i really need to go on a better following spree here. Who is working on llm agents and/or MARL?

Can a Large Language Model (LLM) with zero Pokémon-specific training achieve expert-level performance in competitive Pokémon battles? Introducing PokéChamp, our minimax LLM agent that reaches top 30%-10% human-level Elo on Pokémon Showdown! New paper on arXiv and code on github!

We are not out of the woods yet... Looks like NVIDIA 5090 stockx prices are starting to stagnate. Likely to be over 4 weeks until under $100 per GB of VRAM unless we are blessed with significantly more stock

Based on stockx resales, NVIDIA 5090 prices might normalize in 2 weeks time

Is GRPO sufficient? Let's look at the OG AlphaGo paper --> Policy network: yes, from LLM Rollouts: yes, from advantage calculation Value network: no Perhaps we are in the REINFORCE era of RL for LLMs still

sometimes it takes more than machine learning to train a model

Telling researchers not to research something is a really great motivation for those whose primary motivation is why not

What if we had alphastar for LLMs soon? Where to start->

RLC deadline is way too close to ICML

Redefining MARL so at least the reviewers and I can argue about something I want to talk about

i just need to freeze experiments and reduce to 8 pages

Yep no 5090 for me

🧠DeepSeek's R1 report highlights challenges in applying AlphaZero to LLMs. Gaming AI offers a key insight: AlphaStar, like an LLM, starts with imitation learning, then RL, and crucially, league self-play. League diversity helps avoid local optima. How do we implement this? ⬇️

keep it 💯