How do LLMs learn to reason from data? Are they ~retrieving the answers from parametric knowledge🦜? In our new preprint, we look at the pretraining data and find evidence against this: Procedural knowledge in pretraining drives LLM reasoning ⚙️🔢 🧵⬇️

We got an 🥂 Outstanding Paper Award!! Cannot be more grateful 🥹 This is super validating for our long pursuit of computational work on QUD. Congrats to the amazing @yatingwu.bsky.social, Ritika Mangla, Alex Dimakis, @gregdnlp.bsky.social

New paper from Martha Lewis and me: "Evaluating the Robustness of Analogical Reasoning in Large Language Models" Preprint: arxiv.org/pdf/2411.14215 This is a much-extended follow-up on our earlier pre-print on "counterfactual tasks" in letter-string analogies. 🧵