My Matters Arising concerning a paper on the legal determinants of terrorism is now out in @nathumbehav.nature.com. The original paper is now retracted. To learn why, read on for a story of irregularities, imputations, and impossible values. 1/x doi.org/10.1038/s41562-025-02347-7

Pleased to announce that my paper on hypothetical bias in experiments is now published in Experimental Economics! If you're interested in (how we figure out) whether real stakes and incentives matter in experiments, give it a read! 🔗: doi.org/10.1017/eec....

Had a great time presenting my job market paper at the Lindau Nobel Meeting in Economic Sciences! 🔗 : osf.io/d7sqr_v1/ #LINOecon #EconSky

Registration closes July 4! Come join us in-person or online in Amsterdam!

Dutch academics, please share - replication games are coming to Vrije Universiteit Amsterdam on July 19! If you live in the Netherlands, or want a good reason to visit, come take a day to network, hone your skills, and potentially coauthor a paper. Registration: www.surveymonkey.ca/r/Replicatio...

#GDRI_rep Update 7: Retraction! Our comment of "Parent–teacher meetings and student outcomes: Evidence from a developing country" has led to the paper being retracted. Our comment is accepted as is. Short 🧵

This will likely be a good initiative, conditional on it not substituting for the journal publishing full comments. Publication is an important incentive for reproducibility/robustness analyses and substantive empirical critiques almost always take more than one page.

#GDRI_rep Update 5: We have a new report. We reproduced the paper entitled "Partisan Effects of Information Campaigns in Competitive Authoritarian Elections: Evidence from Bangladesh" by Ahmed, Hodler and Islam published at the Economic Journal. See below for links to report and authors' responses.

What academic journal should I start? Wrong answers only

#GDRI_rep Update 4: We have a new report. We reproduced the paper entitled "Parent–teacher meetings and student outcomes" by Islam @ European Economic Review. See below for links to report and author's response.

Thank you @carlislerainey.bsky.social for boosting my job market paper! I’’m a fan of your prior work on this topic. If you want to learn more, check out the paper and my BIBAP seminar on it tomorrow, 3/12 7 AM CET/5 PM Sydney time. www.unsw.edu.au/business/beh...

#5 Here's another recent critique of close elections RDDs. It's unclear whether there is actually good evidence against potentially fatal precise manipulation around the cutoff in these designs.

I'll be waking up early (7 AM CET) on Tuesday, March 12 to present my job market paper at 5 PM Sydney time! If you're awake too, stop by to hear me talk about equivalence testing, replication-based methods research, and the robustness of null results in economics!

Relatedly, I also develop new equivalence testing methods for testing running variable (RV) manipulation in RDD. In 36 RDD publications, I find >44% of RV density discontinuities at treatment cutoffs can't be significantly bounded beneath 50% upward jumps. 🧵: bsky.app/profile/jack...

#GDRI_rep Update 2: I4R sent a report to the original authors for the PLOS One article "“Food insecurity and mental health of women during COVID-19: Evidence from a developing country". All authors have now emailed PLOS One to withdraw their names. osf.io/7tzek/

#GDRI_rep Update 1: I4R are now reproducing all published papers that use data from GDRI, or are closely related in other ways. Here is a first update on our work. 🧵

Our proposal for post-publication review: “our system would add only 1 percent to the total reviewing effort, while providing important perspectives on papers representing more than one-quarter of the citations received by these influential journals.” statmodeling.stat.columbia.edu/2025/02/26/pp/

The Need for Equivalence Testing in Economics

As a coauthor on the AEJ:AE report, it’s a lot. Oh my god, It’s a lot. In that paper, outcomes are inconsistently handled both in the code and in the field, the paper’s data is connected to a bunch of other experiments, and we find irregularities in the raw survey files.

I had a wonderful time presenting my job market paper at RWI Essen! Thank you @jrgptrs.bsky.social and Julian Rose for hosting and organizing!

New working paper! @i4replication.bsky.social does a good job detailing the experiment’s results, so for anyone who considers LLMs/AI as a (soon-to-be) solution to reproducibility in social science, let me walk you through the 🔥hell🔥 of trying to reproduce a paper on an "AI-led" team. 1/7

New paper for holiday reading! @isager.bsky.social and I provide an introduction to three-sided testing, a framework for testing estimates' practical significance. We offer a tutorial, Shiny app, + commands/code in #Rstats, #Jamovi, + #Stata. 1/9 osf.io/preprints/psyarxiv/8y925 #EconSky #PsychSky

New preprint: “Three-Sided Testing to Establish Practical Significance: A Tutorial” (osf.io/preprints/ps...) with @FitzgeraldJack_, proposing an upgraded version of the TOST equivalence test 🧵

A huge thanks to @sweiwang.bsky.social for highlighting my job market paper, The Need for Equivalence Testing in Economics!

In more recent news, I thoroughly enjoyed presenting The Need for Equivalence Testing in Economics at the Netherlands Reproducibility Network Symposium and Platform for Young Meta-Scientists Symposium, with great discussion from Tsz Keung Wong!

Two weeks ago, I had a wonderful time presenting The Need for Equivalence Testing in Economics at the Leibniz Open Science Day! (pic: @prashantgarg.bsky.social)

Moving from Florida to the Netherlands is a trip because for three months of the year, you can look to the sky and think ‘wow, what a beautiful sunset’ before realizing it’s 2:30 in the afternoon

Suppose I say “I regressed V1 on V2.” Which one do you assume is the dependent variable? 1: V1 2: V2 #EconSky #rstats #statistics

Super important resource for the field, give them a follow! #EconSky

I had an excellent time presenting this paper to the Behavioural Insights for Business and Policy Network at the University of New South Wales. A huge thanks to @impartialspectator.bsky.social for hosting!

Do real stakes/incentives matter in experiments? Recent studies say they don’t. My new paper shows that these studies’ results — and those of most hypothetical bias experiments — are uninformative when we care about treatment effects. 1/19 #EconSky #PsychSky #PoliSky 🔗: papers.tinbergen.nl/24070.pdf

Here is the starter pack for folks in science of science, meta science, STS, philosophy of science, history of science, economics of science, science policy, AI+science, and so on. Feel free to nominate and self-nominate! #SciSci #STS #MetaScience #SciPol #AI4Science #ScAISci go.bsky.app/2R1E4TD

A research highlight for #EconSky! I design equivalence tests for running variable (RV) manipulation in regression discontinuity (RDD), show that RV manipulation can't be ruled out in lots of published RDD research, and offer the lddtest command in Stata/R. 1/x 🔗: hdl.handle.net/10419/300277

PSA to Dutch academics: It looks like the protest has been cancelled at the last minute. nos.nl/artikel/2544...

Protests are better together! If you're headed to the demonstration tomorrow in Utrecht to protest government cuts to higher education and research, and you'd like to meet up, feel free to reach out!

@o.simardcasanova.net has curated an excellent starter pack of folks working on economics of science and replication! bsky.app/starter-pack...

A great conversation about my experiencing publishing a replication with PNAS and about replication work more generally. This is a podcast worth tuning into!