Planning my next conversation starter

I too was very glad to see this! But I feel like the whole episode bodes badly for the future. It’s not sustainable to rely on the CEO of a private company to act against their financial self-interest in order to curtail high-risk AI deployment (here mass surveillance and fully autonomous weapons).

🔔 “How real is the LLM threat to online research in academia?” will be live today. Experts from Microsoft Research, MIT / Stanford, Max Planck Institute, and Prolific discuss the threat of agentic AI to online research, and how to protect against it. Link to join live below. #AcademicSky #Research

Reposting for visibility. Many researchers still appear oblivious to this fact, which is terrifying! It should be included in every experiment design 101.

When I pitch academics on my paper on nulls one common and understandable reaction is "but they're probably noisy and thus uninformative nulls." This is true, but it misses the key realization that WE PUBLISH THE RESULT WHEN THE NOISY TEST IS P<0.05.

I have a new paper. We look at ~all stats articles in political science post-2010 & show that 94% have abstracts that claim to reject a null. Only 2% present only null results. This is hard to explain unless the research process has a filter that only lets rejections through.

@benmtappin.bsky.social I just was pointed to this which is much more thorough and arrives at the same conclusion: www.exponentialview.co/p/how-95-esc... The "95% fail" number is essentially meaningless

Felix and friends looking closely at the details so you don’t have to 👌👇

A short note on questionable AI studies or why friends don’t let friends make %-claims based on small-n qualitative research interview reports New week, new AI newsletter from Marina and myself here at RISJ: buff.ly/ckaUSn9

It’s that time of year again

Interesting new paper in Political Psychology from @benmtappin.bsky.social and Ryan McKay investigating party cues onlinelibrary.wiley.com/doi/10.1111/...

🚨New WP "@Grok is this true?" We analyze 1.6M factcheck requests on X (grok & Perplexity) 📌Usage is polarized, Grok users more likely to be Reps 📌BUT Rep posts rated as false more often—even by Grok 📌Bot agreement with factchecks is OK but not great; APIs match fact-checkers osf.io/preprints/ps...

My Centre is a unique place to do a PhD in Philosophy, because you can be in constant contact with experts in veterinary medicine, psychology, zoology and policy and be part of a team united by a shared interest in animal minds. We now have our 1st ever PhD scholarship: www.lse.ac.uk/sentience/phd

📢 JOB ALERT! Postdoc opportunity in Political Behaviour & Political Economy (UKRI‑funded) - Please do share with anyone who might be a great fit. If you’re interested or would like to know more, please feel free to get in touch!! jobs.reading.ac.uk/Job/JobDetai...

🎺 Call for proposals 🎺 1️⃣ replicate an existing experiment 2️⃣ run a novel experiment on repdata.com 3️⃣ coauthor with Mary McGrath and me to meta-analyze the replications and existing studies 4️⃣ publish your study details: alexandercoppock.com/replication_... applications open Feb 1 please repost!

I land somewhere between this and the OP. Evaluating the quality of the methods often requires fully understanding the research question and estimand. And that usually requires reading the intro. (Disclaimer: but even then it’s no guarantee😭 cf. www.the100.ci/2024/08/27/l... @dingdingpeng.the100.ci)

My students are in for a treat next week

A very interesting data-rich analysis of persuasion on digital media by @benmtappin.bsky.social. Recommend! open.substack.com/pub/benmtapp...

🚨 New in Nature+Science!🚨 AI chatbots can shift voter attitudes on candidates & policies, often by 10+pp 🔹Exps in US Canada Poland & UK 🔹More “facts”→more persuasion (not psych tricks) 🔹Increasing persuasiveness reduces "fact" accuracy 🔹Right-leaning bots=more inaccurate

🚨 New working paper 🚨 We often see populist parties like Reform UK blame higher energy bills on climate change policies. What are the political consequences of this strategy? Very early draft; comments and criticisms are welcomed! full draft: z-dickson.github.io/assets/dicks...

"While testing one dimension at a time can yield simple results, those effects may not generalise to richer, real-world contexts." Read our new POAL Methods Briefs on Conjoint Experiments from Thomas Robinson! Link: www.poal.co.uk/research/met...

Insightful long-read: "With AGI [artificial general intelligence], powerful actors will lose their incentive to invest in regular people–just as resource-rich states today neglect their citizens because their wealth comes from natural resources rather than taxing human labor." intelligence-curse.ai

🗞️ 🤖 Weekend reading anyone? For the launch of @transformernews.ai as a standalone publication, they invited me to contribute a piece on what persuasive AI might mean for democracy and elections. Here’s the result… buff.ly/OJsNmpK

WE ARE HIRING! 2 Lecturers in Quantitative Social Science. Want a friendly interdisciplinary department in one of the world's most vibrant cities? This just might be for you. Apply by: 10 Oct www.ucl.ac.uk/work-at-ucl/...

Wrote about how the UK's online age verification requirements are already proving to be a disaster (as UK regulators were clearly warned it would be) and how unhelpful the UK's response to this mess has been, including their tech minister saying anyone who complains supports predators.

The largest investigation of AI persuasion with 76,977 participants across 3 large-scale experiments. Excellent work by @ox.ac.uk PhD candidate @kobihackenburg.bsky.social. 19 LLMs. 707 political issues. 466,769 fact-checkable claims evaluated. arxiv.org/abs/2507.13919 #AcademicSky #MLSky #PhDSky

I am so proud of the brilliant @oii.ox.ac.uk DPhil @kobihackenburg.bsky.social and this wonderful “bees-knees” paper on conversational AI and political persuasion @AISecurityInst - it is a “must-read”, comments welcome!

A great post, this point especially: "I don’t mean to argue all research needs to be slow and fully documented. When we are just starting in a new area, it’s chaos. But at some point, by the time results are reported, the workflow needs to be professionalized. Research is not a hobby. It’s a job."

VERY excited about this massive AI persuasion experiment - big effects on UK policy positions, driven mostly by post-training (rather than hyperpersonalization or model scale); and the more info the model provides, the more persuasive it is

Exciting new research by @benmtappin.bsky.social and colleagues ⬇️

👇New experiments in which we aimed to map the levers and scope of political persuasion with conversational AI models. It was a tremendous privilege to lead on this work alongside the brilliant @kobihackenburg.bsky.social. The paper is packed with results and we'd love your comments!

Have now read this paper in detail. It’s a tour de force. If you’re interested in the potential impact of AI on election outcomes you should put it by your bedside. Key takeaway: let’s remain alive to, but healthily skeptical of, the possibility of large impacts: knightcolumbia.org/content/dont...

New paper in PSPB! journals.sagepub.com/doi/10.1177/... Well, actually, not "new". We first put this paper online way back Dec 2022... in any case, we think it's really cool! We find that conspiracy believers tend to be overconfident & really don't seem to realize that most disagree with them

New Substack post It's very personal: my story of a 20-year academic career, and the many challenges of theoretical and cross-disciplinary work As I put it in the subtitle: There is a lot of success and a lot of pain here, and no happy ending thomscottphillips.substack.com/p/happy-in-t...

I think the current state of social science research is pretty bad and I wrote something for @asteriskmag.bsky.social about it. asteriskmag.com/issues/10/ca...

Some snapshots from our event on Friday, where we discussed the future of polling, the challenges ahead, and how AI is shaping survey research. Massive THANK YOU to all panelists for sharing their wisdom with us! Here’s a thread with some key take-aways:

Thanks to everybody who chimed in! I arrived at the conclusion that (1) there's a lot of interesting stuff about interactions and (2) the figure I was looking for does not exist. So, I made it myself! Here's a simple illustration of how to control for confounding in interactions:>

🚨New R package! {easymediation}🚨 The *Simplest* and *Most Correct* Way to Do Causal Mediation Analysis Are you tired of explaining mediation analysis to your colleagues? Just send them this package. github.com/rpsychologis...

PSA: If you want to compare whether an association varies between groups (i.e., you're interested in an interaction) but you also note that the groups differ in some variable that could "explain away" any interaction -- then you need to interact that variable with the predictor to control for it.

It’s high time we started asking the important questions

Interesting 👇

"Deep getting to know" 😂

New visualization tool alert! The vayr package version 1.0.0 is now on CRAN. It contains position adjustments for ggplot2 that help with overplotting in pleasing ways. My favorite is position_sunflower(). - install.packages("vayr") - alexandercoppock.com/vayr #rstats #ggplot2 #dataviz

BlueSky's stats meme game remains healthy

What a line "we argue that our evidence is consistent with finding that badass TikTok edits are approximately as effective as professionally produced television ads" doi.org/10.1177/2056...

“I am the last person to tell people what they should be interested in—but the first one to tell them that if they don’t tell me what they are trying to estimate in the first place, why even bother.” 🤙

Thank god at least SOME firms out there are asking the important questions about political differences

If you’re interested in actual numbers, Hannah Ritchie has a post on the topic www.sustainabilitybynumbers.com/p/ai-energy-... and Wikipedia has a list of consumption by country: en.m.wikipedia.org/wiki/List_of...

Proudly presenting the (for now) final version of "Why experiments work." To share the materials in a slightly more professional manner, I added a "Resources" page to my website: juliarohrer.com/resources/. That was long overdue anyway; now there's also a curated list of my papers and blog posts.