New paper: "EconEvals: Benchmarks and Litmus Tests for LLM Agents in Unknown Environments" We construct economic environments to measure the capabilities and tendencies of LLMs and LLM agents in pricing, procurement, task allocation and more. 1/6

This is part of a super interesting line of work. For this paper I got to help as a member of an AI-led team, the most fun treatment 🙂 (I may be biased). Current LLMs often "get stuck" on complex tasks like reproducing a paper, but we can expect some of these limitations to go away with time.

🚨Major new version🚨 Algorithmic Collusion by Large Language Models Joint w/ @sarafish.bsky.social & @ranshorrer.bsky.social LLMs are automating many business decisions. Pricing might be next (or is already). What if multiple firms, in good faith, to use off-the-shelf-LLMs for pricing? 1/3 #EconSky