Wow! Honored and amazed that our reward models paper has resonated so strongly with the community. Grateful to my co-authors and inspired by all the excellent reward model work at FAccT this year - excited to see the space growing and intrigued to see where things are headed next.

Reward models (RMs) are the moral compass of LLMs – but no one has x-rayed them at scale. We just ran the first exhaustive analysis of 10 leading RMs, and the results were...eye-opening. Wild disagreement, base-model imprint, identity-term bias, mere-exposure quirks & more: 🧵

Just saw that Andrew Barto and Richard Sutton have won the 2024 Turing Award, roughly the computer-science equivalent of the Nobel. Incredibly highly deserved to these two pioneers of reinforcement learning. awards.acm.org/about/2024-t...