I wrote about my experience tinkering with AI for social science research: open.substack.com/pub/ey211/p/...

New paper out at AJPS: "The limits of AI for authoritarian control." The more repression there is, the less information exists in AI's training data, and the worse the AI performs.

We've updated the localLLM package on CRAN (cran.r-project.org/package=loca...). It allows you to run LLMs locally and natively in R. Everything is reproducible. And it's free. Some functionalities on reproducibility and validation🧵👇

Russia, Venezuela, Iran, China, the Sahel region, the United States ... Want to know why state agents carry out brutal repression — or participate in illegal coups? Our new book "Making a Career in Dictatorship" provides answers — it just got published by @academic.oup.com: tinyurl.com/ystwm3tf

Why the slow uptake in political science?

So, randomization is not a *sufficient* condition for good research. Far from it. The best experimental social science being done is that work where either the theory or the operationalization (or both) are the emphasis. Randomization is "easy" - the challenge is what you randomize and why.

Can Large Multimodal Models (LMMs) extract features of urban neighborhoods from street-view images? New with Paige Bollen (OSU) and @joehigton.bsky.social (NYU): Sometimes, but the models better recover national assessments that local ones, even w/additional prompting (which can make things worse!)

New paper: LLMs are increasingly used to label data in political science. But how reliable are these annotations, and what are the consequences for scientific findings? What are best practices? Some new findings from a large empirical evaluation. Paper: eddieyang.net/research/llm_annotation.pdf

Great analogy to connect AI to many canonical political science questions. Political behavior has led the way in studying AI. Excited to see institutions catch up😀

If no resource constraint, what open-weight LLM would you use in your research (for data labeling, coding etc.)?

Awesome work! Love to see different approaches to this problem.

Really interesting read. Refreshing perspective.