1/ Can AI help researchers check whether published social science results actually reproduce? In our new PNAS paper, we tested this directly in the AI Replication Games: 288 researchers, 103 teams, and real replication packages from quantitative social science.

#GDRI_rep Update 10 The orphan article has finally been retracted! Took more than a year, but at last. Read about our investigation (bsky.app/profile/i4re...) and updates ( #GDRI_rep). A good summary: a-ortmann.medium.com/the-asad-isl... and a-ortmann.medium.com/tallying-the...

🧵1/ Our first meta-science paper (with 350+ coauthors) is published today in Nature. It presents one of the largest-ever reproducibility projects in economics & political science. Here’s what we found 👇

Fotbollsallsvenskan börjar om en vecka. Som vanligt blir jag förvånad över att många inte klarar att skilja orsak och verkan, och att statistiken ofta övertolkas. Så här är mitt perspektiv på frågan: Hjälper det att sparka tränaren?

📢 New Working Paper: “Labor Supply under Temporary Wage Increases: Evidence from a Randomized Field Experiment”, with Mats Ekman and @kotsadam.bsky.social 🔭 arxiv.org/abs/2602.11992

Kom och jobba med mig i Karlstad!

🚨 Our comment on Begum, Grossman & Islam (2018) is now out in Demography @readdemography.bsky.social (w/ @carlbonander.bsky.social, Bensch, @nikljako.bsky.social & Brodeur @i4replication.bsky.social): doi.org/10.1215/0070...

Dagens utdelning från Vetenskapsrådet (VR) till Sveriges nationalekonomer, endast två anslag om totalt 12 miljoner kronor. Så inget trendbrott jämfört med Olles och min granskning. www.tidningencurie.se/debatt/missg...

#GDRI_rep Update 9: We have been waiting a couple of months to make two announcements. A few journals were in the process of retracting GDRI article. The editors informed the authors and then lawyers got involved. Who knows how long it will take.

🚨 Our comment on Begum, Grossman & Islam (2018) in @readdemography.bsky.social is now available as an @i4replication.bsky.social working paper (w/ @carlbonander.bsky.social, @nikljako.bsky.social, Bensch & Brodeur). Read it here 👉 www.econstor.eu/bitstream/10...

#GDRI_rep Update 8a: Major updates coming up over the next few weeks. 1st update: We reproduced "Parent–teacher meetings and student outcomes" some time ago. The paper was then retracted by the editors of the Euro Econ Rev. Additional studies are connected (see diagram) 🧵

🚨 REPLICATION REPORT UPDATE: One year ago, a tweet by John Holbein alerted me, @ollefolke.bsky.social, and @jopieboy.bsky.social to a paper with a shocking result about Sweden’s law criminalizing the purchase of sex.🧵

A re-analysis of Ciacci's (2024) "Banning the purchase of sex increases cases of rape: Evidence from Sweden" Journal of Population Economics reveals major issues. A year ago, reproducers Adema, Folke, and Rickne found coding errors driving the paper's key results. Let's unpack this in a 🧵

Where we stand on the Islam-GDRI-Monash saga. A tally and some questions ... a-ortmann.medium.com/tallying-the...

More traction for the Islam - GDRI saga. Monash claiming that he “holds no official position with GDRI and has not received any personal or material benefit from GDRI since its inception in 2010”. Hmmmh. www.theaustralian.com.au/education/mo...

A Bangladesh-based organization focused on development economics and its founder have been churning out papers filled with misstatements, inconsistencies, ethical lapses and “statistically improbable data,” according to researchers involved in an ongoing effort to replicate the work.

The Islam - GDRI saga is gaining some traction. retractionwatch.com/2025/04/10/r...

Fantastic coauthors made this forensic data investigation fire!

The original paper is now retracted and our comment accepted for publication in the European Economic Review 🔥

I am very happy that the editors took our report seriously. Hope more retractions will follow.

#GDRI_rep Update 6: Some of the authors have responded to our report on "Raising Health Awareness in Rural Communities" @ The Review of Economics and Statistics. Authors received our report Feb 6. A short 🧵

Excellent find. How do you do this organizationally? How many enumerators do you need? How do you pay them? How do you get respondents to respond so well that your attrition rate over four waves is only 1.7%? The authors are rather shy on these issues and the costs. Clearly, GDRI worked magically.

Some would probably say it is impossible. Or maybe everyone?

This gem is hidden in the response to the response. ”…within a few hours and under extreme time pressure 11,843 households were visited and 22,686 respondents interviewed (two spouses in each household).” Indeed hard to believe 🫠

Since we started mass reproducing GDRI, we have requested several replication packages. We have not managed to get a single new replication package as of today (excluding updated packages for one article at the request of editors).

#GDRI_rep Update 5: We have a new report. We reproduced the paper entitled "Partisan Effects of Information Campaigns in Competitive Authoritarian Elections: Evidence from Bangladesh" by Ahmed, Hodler and Islam published at the Economic Journal. See below for links to report and authors' responses.

Bli min kollega! Denna gång söker vi en statistiker (eller liknande) som gärna får jobba både i nationalekonomi och statistik.

In a new @i4replication.bsky.social report we replicate Islam (2019). Main issue: the randomized controlled trial was not randomized. And many other things on top of that. See full report for details. (w/ @carlbonander.bsky.social, @nikljako.bsky.social, @flxhlzmstr.bsky.social, Bensch & Brodeur)

#GDRI_rep Update 4: We have a new report. We reproduced the paper entitled "Parent–teacher meetings and student outcomes" by Islam @ European Economic Review. See below for links to report and author's response.

#GDRI_rep Update 3: The paper "Improving Hygiene and Sanitation through Parental Skill Training" by Asadul Islam, Umair Khalil and Tabassum Rahman was conditionally accepted at Economic Development and Cultural Change. We requested a replication package. It has now been withdrawn

We have a new report out about the #GDRI_rep. The paper has not been retracted but all three authors have withdrawn their names.

#GDRI_rep Update 2: I4R sent a report to the original authors for the PLOS One article "“Food insecurity and mental health of women during COVID-19: Evidence from a developing country". All authors have now emailed PLOS One to withdraw their names. osf.io/7tzek/

Kom och jobba hos mig! Jag behöver minst en forskningsassistent, heltid i ett år, tillträde snarast. Meriterande med dokumenterad erfarenhet av: * Enkäter/intervjuer * Statistisk/geografisk analys * Kunskap brottslighetens/otrygghetens geografi * Brottsdata web103.reachmee.com/ext/I005/101...

#GDRI_rep Update 1: I4R are now reproducing all published papers that use data from GDRI, or are closely related in other ways. Here is a first update on our work. 🧵

Fantastic project where I participated. Lots of work from many involved researchers. And interesting findings on the effects of data cleaning on results.

Very impressive replication work - sad outcomes. I suspect this is what happens when research efforts are driven by career concerns rather than by curiosity.

I4R doing the lord’s work. Hopefully these will be a collective wake up call to be more careful and transparent about complicated studies in applied micro (and elsewhere too)

Let’s take a moment to thank all our amazing replicators: Jörg Ankel-Peters, Juan Pablo Aparicio, Gunther Bensch, Carl Bonander, Nikolai Cook, Lenka Fiala, Jack Fitzgerald, Olle Hammar, Felix Holzmeister, Niklas Jakobsson, Anders Kjelsrud, Andreas Kotsadam, Essi Kujansuu, ...

The new practice that economics journals require sharing of data is already bearing fruit. This practice represents an important improvement for the profession.

The Institute for Replication is doing such crucial work in scientific self-correction. A team of 15 people worked pro bono for 3 months to meticulously comb through a series of questionable published findings. We need a better system to reward this type of work.

The work carried out by @i4replication.bsky.social is truly admirable, now I'm very curious to see what the repercussions of scientific misbehavior are going to be

The craziest thing about it is that if you want to believe the most “positive” interpretation, you have to agree that they’re bad at RCTs. Messy, don’t follow protocols, etc. Yet they consistently get the biggest effect sizes! It’s almost like a proof by contradiction.

Reported treatment effects are GIGANTIC. 2.5 sd in India! Never seen anything that large before.

After being alerted about possible misconduct, the I4R are reproducing published papers that use data from a specific NGO (GDRI). This thread releases the first 2 reports and provides more information about the work and responses/statements from authors journals and journals. 🧵

Sunday 9PM ET / Monday 3AM CET / Monday 1PM Melbourne! See you all then :) #EconSky

Abel: I am writing in my capacity as Chair of I4R. Last year, I was contacted by a researcher that alerted me of potential scientific misconduct in a set of papers all using data from a specific NGO.🧵

A new study where we used IV to make patient-level estimates from RCTs with different participation rates comparable: academic.oup.com/europace/adv...

We are pleased to to announce that enrolments are now open for the “Panel Data for Causal Research Designs” course co-organised with SIdE Italian Econometric Association. 📍Where? Bertinoro 📆 When? 29 June – 5 July 2025 Click the link below for further details ⬇️ irvapp.fbk.eu/news/detail/...

New research alert! Our study investigates the effectiveness of human-only, AI-assisted, and AI-led teams in assessing the reproducibility of quantitative social science research. We've got some surprising findings!