My former MS student Chris Cui (now PhD student with @rajammanabrolu.bsky.social)motivates Text Adventure Games as testbeds for reasoning. Provides a new benchmark suite of text games. Observes that Zork still kicks LLM’s butts despite training on walkthroughs arxiv.org/abs/2504.14128

My entire PEARLS Lab, and many NVIDIA colleagues, will be at #neurips2025 to chat about their latest. Some papers accepted to the conf are already outdated so just reach out to. Thread 🧵

I am extremely honored and humbled to have been awarded a Test-of-Time award for my 2005 paper "From Linear Story Generation to Branching Story Graphs" with R. Michael Young

I've done a few versions of this talk but this is the first that's been recorded publicly, thanks to IVADO Montreal A good overview of things my lab has been up to in the last year or so at least in balancing safety/capabilities of (embodied) AI Agents www.youtube.com/watch?v=S-kV...

🔥Excited to share our new work: "A Practitioner's Guide to Multi-turn Agentic Reinforcement Learning"! We study what actually works for agentic multi-turn RL with varying 🌎Environment, 🤖Policy, and ⭐Reward. We conduct various ablations and empirical analysis on 🧩TextWorld, 🧙ALFWorld, and 🧑‍💻SWE-Gym.

I'll be at #CoLM2025 and the IVADO agents workshop right before in Montreal. My students will be presenting two papers in the main conf. I'll also do a ws keynote where I'll talk about some of our latest. Come by and say hi next week!

I'm probably mostly going to stop posting on this site. There's close to no engagement and it's not worth the effort to cross post for the amount of time that takes. Find me elsewhere / email me

I recently left Mosaic/Databricks Research. It's been a ride building out the RL team from <4 ppl to 20+ across two companies & acquisition +figuring out RL as a Service in prod. Mosaic had insane talent density Some "relaxation" while I put out Prof fires for a smol bit then new adventures!

If you work in the intersection of NLP and games/narrative, then this workshop is for you! wordplay-workshop.github.io/cfp/ Organized by the amazing @laramartin.net and @rajammanabrolu.bsky.social (among others)

The thing that feels so off about the core tech world is that every convo is very transactional. Maybe true elsewhere too. "Oh you're an expert in RL, can you answer questions about my new startup?" Every single (Bay) party. No I do not want to consult. I just wanna hang out.

Of all the labeling startups out there to acquihire, this was... an interesting choice. Says a lot actually

. @bosungkim.bsky.social will be at #CVPR2025 in Nashville this week to present this and just generally talk about scaling memory for embodied agents! Catch her at the poster sessions and also the Foundation Models meets Embodied Agents Workshop on Wed

I've heard this personally from multiple PMs at AI companies. Students are one of the biggest demographics and they need to "break in" and have even more usage to improve their metrics. Classic corporate economic incentives

Tis the era of bringing back every AI benchmark ever but this time by the LLM people and for the LLMs

Had a fun little visit to Cambridge LTL where I talked about a bunch of my lab's latest papers including some still not public with the key takeaway that "RL can absolutely learn new things and is not just resurfacing knowledge" talks.cam.ac.uk/show/archive...

Interesting tidbit from UCSD's Victor Shih on a podcast talking about Chinese AGI efforts is that Deepseek is good at Chinese govt doc understanding cause that's what affects stock prices most and DS is a hedge fund. www.youtube.com/watch?v=b1Te...

Looks like Gemini gets AIR 6 in #JEE2025 with a score of 323 Only 5 highschoolers in all India do better than an LLM in the single most important exam of their to get into the IITs The legacy edu selection systems are now worse than useless

I get prepping for worst case scenarios but a lot of AI Safety debates I somehow end up these days in boil down to "assume you have Machine God in a box, now tell me how to align it" I could rant for hours but seriously y'all this isn't productive

"Foundation" models for embodied agents are all the rage but how to actually do complex looong context reasoning? Can we scale Beyond Needle(s) in the (Embodied) Haystack? ∞-THOR is an infinite len sim framework + guide on (new) architectures/training methods for VLA models

I'm presenting today on AI Agents vs Agency Law at the 12th Governance of Emerging Technologies and Science Conference events.asucollegeoflaw.com/gets/ If agency law were to be applied to AI Agents (mostly in ecommerce settings), where does current AI align with the laws and where does it not?

I like the Ultra Scale Playbook from Huggingface and give it to my MS/first year PhD students to read as a prereq huggingface.co/spaces/nanot... Is there a "RLSys" version of this on scaling RL+LLM training? If not + there's OSS community interest, I'll prob write one?

that's what's hot

We now have a whole YouTube video explaining our MINDcraft paper, check it out! youtu.be/MeEcxh9St24

The part of the Prof job I'm enjoying by far the most right now is teaching actually

This is reasonably written and echoes many of my own fears. The upside of AI is too huge to pass up but also there's a high chance that the vast majority of humanity is on track to becoming economically obsolete without really any transition plan www.theguardian.com/books/2025/m...

What's with these arguments over whether X or Y or whatever was the first LLM RL library? These all came in the last 3 months We wrote multiturn RL4LMs like 3+ years ago github.com/allenai/RL4LMs There were other simple versions even before. ML ppl approaching goldfish memory

Cool looking paper from @markriedl.bsky.social's lab arxiv.org/abs/2505.03547 making significant improvements over our arxiv.org/abs/2001.10161 from 5+ years ago

Yeah this is actual unironic advice for any 1st years: you seriously need to go get beers or otherwise hangout socially with more senior grad students (no one else will really know) and hopefully get ensconced enough that someone will tell you “hey just fyi Dr. GoodPubs is low-key a total sociopath”

Some progress! Looks like o3/4 get about 16% on the human written portion of TALES compared to previous SoTA of 13%

Fascinating read as an outsider to *CL all grad school www.quantamagazine.org/when-chatgpt... Things I hear a lot "no one thought next token pred would work, or RL on language, etc". But that's just not true, there were def ppl working on them (just weren't taken seriously by some communities)

A very recent paper confirms a hypothesis: the time delay between what's common knowledge in corpo LLM world and "public" knowledge is actually increasing

The future of embodied AI revolves around *collaborative* multi agent scenarios that need natural language communication, task delegation, resource sharing, and more ⛏️ Here are MINDcraft and MineCollab, a simulator and benchmark purpose built to enable research in this area!

It's *PO'clock, this time IRPO In-Context Ranking Policy Optimization! An RL algorithm inspired by trad retrieval that trains agents to more effectively use lists of documents in context for better multi-hop {QA, agentic tasks, and more}!

Introducing TALES - Text Adventure Learning Environment Suite A benchmark of a few hundred text envs: science experiments and embodied cooking to solving murder mysteries. We test over 30 of the best LLM agents and pinpoint failure modes +how to improve 👨‍💻pip install tale-suite

Excited to see y'all in Singapore soon. Schedule almost fully packed already 🫡

The Wordplay Workshop is back! 5th edition with EMNLP in Suzhou this Dec. We're also hosting a competition this time on making more realistic LLM powered NPCs in games! As always come by and chat all things text agents! wordplay-workshop.github.io www.aicrowd.com/challenges/c...

5.2-6 earthquakes epicentered about 60 miles east of SD. Can def feel it in UCSD CSE. Ppl on campus should monitor Triton Alerts closely for after shocks earthquake.usgs.gov/earthquakes/...

bro did you see 2b at 20b bro. no bro did you see did 3b at 32b bro actually we also did 10b at aaaaaaaa bro 500k over 5 years bro wow with 58.5% overhead bro between 4 PIs bro so blessed aaaaaaaa

Apropos of nothing I highly recommend the Fall of Civilizations podcast. Deep dives into histories of great civs and their falls. The only podcast I really listen to fallofcivilizationspodcast.com

New age VLM agents are a security nightmare in general. This is all stuff cybersecurity ppl have known for ages but is new for Big Model gang Unclear how all the companies have nominal safety teams and write blogs about risk yet still allow deployments like this

Just hung out with some ppl who run my fav boba place in SD. My fav teas are going off the menu cause supply chain + profitability issues now. I am actually ready to cry 😭

This was a shit post btw if that wasn't clear, the correct answer is obviously async

2T policy, 2T critic, 2T reference, 2T reward It is time to go back to REINFORCE lol

👀 soon youtu.be/iDJ6GrHNoDs

Looks like DeepSeek is also now on the Inference time scaling for reward models train arxiv.org/abs/2504.02495 with a cute diagram comparing it to our CLoud rewards arxiv.org/abs/2408.11791

An early neural story generator (work with @laramartin.net) once generated the sentence "the horse proves itself as a lawn chair entrepreneur". In retrospect, that story was very good and completely indistinguishable from reality in 2025.

Tfw you make a new course from scratch and the final course evals from students are overwhelmingly positive. I could get used to this 🥹

Two years ago at Mosaic we had an idea for an "RLXF as a service" One year and an acquisition later, the prototypes went into preview at Databricks Today we share some results and findings of "what does it take to actually do enterprise RL and put it into a real product"

Reading for the day. An intersection of law and AI Agent space from @markriedl.bsky.social arxiv.org/abs/2502.18359

Give me two years to build ASGI and I'll spend the first year building ASGI Evals. --Mecha Abe, 2025 (probably) Very excited to share a bunch of new papers with this design principle from my lab soon!