Mark J. Nelson (@mm-jj-nn.bsky.social)

Interesting new agent benchmark from MSR, DELEGATE-52, that "simulates long delegated workflows that require in-depth document editing across 52 professional domains". Main finding is that even frontier models, for the modeled workflows, end up corrupting large parts of documents along the way.

loading . . .

LLMs Corrupt Your Documents When You Delegate Large Language Models (LLMs) are poised to disrupt knowledge work, with the emergence of delegated work as a new interaction paradigm (e.g., vibe coding). Delegation requires trust - the expectation t... https://arxiv.org/abs/2604.15597

5 days ago