New Nature podcast episode about ERROR and the Perspectives on Scientific Error workshop!

“We pay experts to examine important and influential scientific publications for errors ... We expect most published research to contain some errors ... our reward system pays bonuses to both authors and reviewers even when minor errors are found ..." statmodeling.stat.columbia.edu/2025/07/13/e...

✨ ERROR (@error.reviews) is a bug-bounty program for science that seeks to estimate the prevalence and nature of errors. error.reviews

EU legislation requiring clothes be reused and recycled may be based on a numerical error in a 2017 NGO report where $460 billion was added instead of subtracted. www.frontiersin.org/journals/sus...

Thank you for sharing this! Reposting to note a @nature.com feature from Aug 2024 by @plantnerd.bsky.social on the ERROR project. 🧪 www.nature.com/articles/d41...

My first time acting as a recommender for the ERROR project. Huge props to the authors @erichehman.bsky.social et al. for exposing themselves to this detailed post-pub peer review and to Will Lowe @conjugateprior.org for his detailed review and reproduction.

Really amazingly heartening to see that academic discourse can also be like this

One of our main goals with @error.reviews is to shift scientific culture to identify, acknowledge, and correct errors in the literature. Eric’s response to the recent report on his work is a perfect example of embodying these values. Kudos to the authorship team and reviewer Will Lowe.

It is an honor for anyone to care enough about your work to check it, so thanks dudes. A mistake I suggest others not make is failing to ensure that all the data, from download, to merge, to analysis is available and each step, not just the final analysis, is reproducible.

This sounds like the same error I blogged about a few years ago, the common error of trying to control for population (or body size or many etc) by dividing the outcome variable by it. Props to the authors for seeking review and taking the issue seriously. Role models for us all.

A ⭐️⭐️⭐️⭐️⭐️ model response to finding problems in your published research. If you’ve published more than a few papers, you too have almost certainly made at least one mistake, so we should normalise addressing them like this! Great to see @error.reviews approaching error detection so sensitively

THIS is how science works. A great and honorable reaction - especially in these times of resistance to facts and smug self-righteousness. Thanks to @erichehman.bsky.social & @error.reviews!

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

New report! Hehman et al. (2018) "Disproportionate Use of Lethal Force in Policing Is Associated With Regional Racial Biases". Based on the review by @conjugateprior.org, we find a Major Error that affects a core conclusion. We recommend the authors seek a correction. error.reviews/reviews/hehm...

ERROR is our scientific bug-bounty program. We pay reviewers and authors of impactful published articles to do a deep dive on results, data, code, etc to check for errors. Our second report of 2025 (and my first acting as Recommender for the review) is out now!

Excited to announce our second report of 2025! Fernbach et al. (2019) "Extreme opponents of genetically modified foods know the least but think they know the most". Based on the review by R Chris Fraley, we find Minor Errors which do not affect core conclusions. error.reviews/reviews/fern...

ERROR is our scientific bug-bounty program. We pay reviewers (and authors!) of impactful published articles to do a deep dive on results, data, code, etc to check for errors. Our second report is out now! More information about the program here: error.reviews

Great to see this initiative providing these insights! Fingers crossed programmes like these don't depend on a few brave souls putting their work forward for scrutiny. I'd like to see this integrated into how funders evaluate the lifecycle of all research they support in the public interest.

We are excited to announce the review of Lades et al. (2020) "Daily emotional well-being during the COVID-19 pandemic". Based on the review by Joop Adema, we find Moderate Errors which do not affect the core conclusions. We recommend the authors seek a correction error.reviews/reviews/lade...

We’re encouraging meaningful commentary with $1,000 rewards for selected PubPeer comments. bsky.app/profile/pubp... Now imagine if @hhmi.bsky.social , NIH, NSF, etc also awarded outstanding public reviews. These could boost careers and CVs while building a vital layer of scientific evaluation.

New paper under review! Danièle Gubler is checking "Environmental sensitivity in children: Development of the Highly Sensitive Child Scale and identification of sensitivity groups" by Pluess et al. for errors. error.reviews/reviews/plue...

New paper under review! @matthewbjane.bsky.social is checking "A meta-analysis of work-family conflict and social support" by French, Dumani, Allen, & Shockley for errors. error.reviews/reviews/fren...

New paper under review! @drewhalbailey.bsky.social is checking "Why are some STEM fields more gender balanced than others?" by Cheryan, Ziegler, @akmontoya.bsky.social, & Jiang for errors. error.reviews/reviews/cher...

“My hope for this is that it goes from something that was unimaginable until it happened, and then it was unthinkable not to do it,” says @ianhussey.bsky.social “You have to give people the permission, and incentive, to think in the first place that errors might exist.” www.wired.com/story/bounty...

Out today in WIRED: "Science Is Full of Errors. Bounty Hunters Are Here to Find Them"

Listen to our own @ianhussey.bsky.social on The Body of Evidence, talking about how to make science self-correcting and why paying bug bounties for errors is cheaper than not doing so bodyofevidence.ca/bounty-hunti...

New paper under review! @frederikaust.com is checking "Patterns of implicit and explicit attitudes: I. Long-term change and stability from 2007 to 2016" by Charlesworth & Banaji for errors. error.reviews/reviews/char...

New paper under review! Florian Pargent is checking "Is romantic desire predictable? Machine learning applied to initial romantic attraction" by @datingdecisions.bsky.social, @pauleastwick.bsky.social, & @elijfinkel.bsky.social for errors. error.reviews/reviews/joel...

New paper under review! Alex Burgoyne is checking "Should we stop thinking about inhibition? Searching for individual and age differences in inhibition ability" by Rey-Mermet, Gade, & Oberauer for errors. error.reviews/reviews/rey_...

Science can be self-correcting — but only if we invest in making it so. Our view on the staggering costs of undetected errors in science, and why funding error detection and correction is less expensive, published today in Nature www.nature.com/articles/d41...

Our paid post publication error detection project @error.reviews reviews has issued its first report! @russpoldrack.bsky.social detected minor errors in @janw.bsky.social's (2018) study on the impact of procedural variations of the go/no-go paradigm. Full report: error.reviews/reviews/wess...

How science should work! Such a cool project 🧮🔍

The first ERROR review has been completed with a decision of minor errors!

We are very excited to announce the first completed ERROR review cycle. Based on the review by @russpoldrack.bsky.social and additional data collection by @janw.bsky.social, we find Minor Errors. error.reviews/reviews/wess...

So happy to see this. I don't think I'm the only postdoc that thinks this would be a dreamy full-time job? (Be paid to thoroughly read fascinating papers in detail, put your critical skills to good use, and contributing to the validity & trustworthiness of science?!)

New paper under review! R Chris Fraley is checking "Extreme opponents of genetically modified foods know the least but think they know the most" by Fernbach, Light, Scott, @yoelinbar.net, & Rozin for errors. error.reviews/reviews/fern...

Today, we are announcing a 3-year research and development project on a new model of scholarly communication called Lifecycle Journals to evaluate its strengths and weaknesses and suitability for transition into a scalable and sustainable solution. Learn more: cos.io/lifecyclejournals

New paper under review! Joop Adema will review "Daily emotional well‐being during the COVID‐19 pandemic" by @lklades.bsky.social, Laffan, Daly, & Delaney error.reviews/reviews/lade...

We are currently looking for a reviewer with expertise in SAS. Domain knowledge in intergroup dynamics would be ideal, but is not a must. If you are interested or know someone who might fit this description, please let us know! Reply here or email: error.reviews/contact/

New paper under review! @steveharoz.com will review "Fake news game confers psychological resistance against online misinformation" by Rozenbeek & @profsanderlinden.bsky.social error.reviews/reviews/roze...

New paper under review! @conjugateprior.org will review "Disproportionate use of lethal force in policing is associated with regional racial biases of residents" by @erichehman.bsky.social, @jkflake.bsky.social, & Calanchini error.reviews/reviews/hehm...

Really exciting new project. I think this is the kind of thing that the public assumes is really common in science - it’s not, yet, but I hope this helps us get there.

Curious to see uptake from authors. Is a bit like the idea of red teams in science (h/t @lakens.bsky.social ), but at a worse timepoint. But you gotta start somewhere! #OverlyCuriousEditor #AcademicSky #ScientificPublishing

A bounty system for catching errors in scientific publications! We definitely need a new sheriff in this particular town.

Error checking is an important service to science, but it doesn't pay the bills. But what if it did? Introducing ERROR, a bug bounty program for science to systematically detect and report errors in academic publications error.reviews