How do we reliably judge if AI companions are performing well on subjective, context-dependent, and deeply human tasks? 🤖 Excited to share the first paper from my postdoc (!!) investigating when LLMs are reliable judges - with empathic communication as a case study 🧐 🧵👇

Super cool opportunity to work with brilliant scientists and fantastic mentors @mattgroh.bsky.social and Dashun Wang 🌟🌟 Feel free to reach out!

Our paper: Decision-Point Guided Safe Policy Improvement We show that a simple approach to learn safe RL policies can outperform most offline RL methods. (+theoretical guarantees!) How? Just allow the state-actions that have been seen enough times! 🤯 arxiv.org/abs/2410.09361