Andreas Krause 4 months ago
SDPO enables RL agents to learn from rich feedback (i.e., not only whether an attempt failed, but why it failed, such as error messages). Even without such rich feedback, SDPO can reflect on past attempts and outperform GRPO. SDPO also accelerates solution discovery at test time!
add a skeleton here at some point