New draft: "Decline effects, statistical artifacts, and a meta-analytic paradox". In this manuscript I show how a common practice in meta-analysis (eg the 2015 Open Science Collaboration) creates artifactual signatures of poor scientific behavior. PDF: raw.githubusercontent.com/richarddmore... 1/x

Simonsohn has now posted a blog response to our recent paper about the poor statistical properties of the P curve. @clintin.bsky.social and I are finishing up a less-technical paper that will serve as a response. But I wanted to address a meta-issue *around* this that may clarify some things. 1/x

I love this post about science and metascience. A lot of quotables but I’ll lead with this: “Those seeking a scientific method – one that can be written down and followed mechanically […] – betray a kind of childish impatience with a process they clearly don’t understand.”

AI slop and the destruction of knowledge irisvanrooijcogsci.com/2025/08/12/a...

1. "'Trusting the experts is not a feature of either a science or democracy," Kennedy said." It's literally a vital feature of both science and of representative democracy. I've written a fair bit about trust in expertise as a vital mechanism in the collective epistemology of science.

We proved they could not scale link.springer.com/article/10.1...

here was our call for methodological standards in metaresearch four years ago. instead of getting fixated on a particular inference we'd like to make, we need to maintain scientific standards, do the hard work, respect the evidence. we can't keep jumping at self-serving solutions without question.

some of the discussion around the p-curve paper is depressing. i still see many missing the point clearly stated in the conclusion, and instead of demanding strong standards or questioning whether they're even asking good questions, they've immediately started asking for replacement methods.

New paper with @richarddmorey.bsky.social now out in JASA, where we critically examine p-curve. Below is Richard’s excellent summary of the many poor statistical properties of p-curve (with link to paper). I wanted to add some conceptual issues that we also tackle in the paper.

New paper by my PhD student @semihaktepe.bsky.social now published 🚀 "Revisiting the effect of discrepant perceptual fluency on truth judgments" tinyurl.com/2eepue5y --> Two experiments & a meta-analysis indicate that high visual contrast does not lead to higher truth judgments.

wherein I'm quoted with "Science is essentially effective to the extent that it can remain independent, decentralized, and democratic." (gift link, email required)

🚀Postdoc position @unimarburg.bsky.social in the project: "Bridging the Gap Between Verbal Psychological Theories & Formal Statistical Modeling with Large Language Models" (funded by @volkswagenstiftung.de) 📅Start: 01.10.2025 | ⏳4 years 🔗 Apply now: uni-marburg.de/jhbCen 🔄 Thanks for sharing!

NEW paper! 💭🖥️ “Combining Psychology with Artificial Intelligence: What could possibly go wrong?” — Brief review paper by @olivia.science & myself, highlighting traps to avoid when combining Psych with AI, and why this is so important. Check out our proposed way forward! 🌟💡 osf.io/preprints/ps...

If you'll be at APS2025 in DC next week, I'll be talking about the terrible statistical properties of the p curve procedure in the "Current Issues in Meta-Science" session Saturday, May 24th at 3pm. This will likely be my last US conference in a very long time. A brief summary follows. 1/

I am looking forward to expanding the scope of my professorship by combining cognitive and statistical modeling with LLMs😊 There will be two job openings for postdoc positions soon - one starting in September 2025 and another one a year later.

Another reminder that this online event will take place this Friday. I might be alone representing the West Coast (it's at 7am PT) but hope colleagues from other time zones consider stopping by and joining the discussion. My talk will be not-so-subtly titled "Replication Is (Not)".

Special Issue Spotlight: Decision research meets #AI. Read a short summary of the special issue from guest editors Ido Erev, Sudeep Bhatia, and @clintin.bsky.social bit.ly/3FkGX0N

Not being anti-fascist at this moment in history is anti-science. Probably always was. But cannot be clearer than these last few weeks.

A visualization I created to share with family and friends about how the NIH indirect rate costs will hurt science AND our economy.

Since everyone is talking about research funding, this perspective by the vice-president of the European Research Council might be of interest. Blue states: Consider joining a multinational research council.

My latest column just appeared in Science, entitled "Free speech, fact-checking, and the right to accurate information”. (www.science.org/doi/10.1126/...) I use one of President Trump’s first executive orders to unpack the terrain between misinformation and claims to free speech 1/n

🚨 New Paper Alert! 🚨 How should scientists decide what to replicate & how? Our latest article explores: 🔍 Book of Truths vs. Book of Conversations – What's the purpose of replication? 📊 Balancing epistemic values, public trust & transparency. www.pnas.org/doi/10.1073/... @clintin.bsky.social

The present and future of peer review: Ideas, interventions, and evidence www.pnas.org/doi/10.1073/...

Dialogues about the practice of science "In this Feature: Should scientists reform the way they carry out research; if so, how? Should we reform the institutions that currently serve as scientific gatekeepers? How do recent technological & cultural revolutions affect the practice of science?"

Now out in PNAS, we provide a conceptual framework for assessing replication decisions and replication reform. Do you view the scientific literature as a "book of truths" or a "book of conversations"? www.pnas.org/doi/abs/10.1... #philsci #science #SciSci #Metascience

Which scientific results should be replicated? Our new PNAS suggests the decision depends on whether you subscribe to “the book of truth” or “book of conversation” @pnas.org @clintin.bsky.social

Now out in PNAS, we provide a conceptual framework for assessing replication decisions and replication reform. Do you view the scientific literature as a "book of truths" or a "book of conversations"? www.pnas.org/doi/abs/10.1... #philsci #science #SciSci #Metascience

How can we make sound replication decisions? www.pnas.org/doi/10.1073/...

New preprint with @winterstat.bsky.social and Ellen Fitzsimmons on ordinal factor analysis, where the latent variables are scaled so that they generally take values from 1 to # of ordered categories. The scaling is intuitive because it is what you use when you treat ordinal variables as continuous.

I'm excited to announce that I am the new editor-in-chief of the Journal of Mathematical Psychology. Please send your best work investigating human behavior with formal approaches. Wish me luck!

🎬🎥🍿 Video of my keynote at MathPsych2024 now available online www.youtube.com/watch?v=WrwN... #CogSci #CriticalAI #AIhype #AGI #PsychSci #PhilSci 🧪

Simonsohn says prereg is only useful to distinguish btw exploratory & confirmatory analyses (a distinction we just wrote against) Gelman: "I disagree, partly bec I think that confirmatory and exploratory statistical analyses are the same thing, partly bec I very rarely care about p-values anyway"

Alright it's here! Where we push back against the exploratory-confirmatory distinction that's become canonized in metascience and talk about what we think exploratory research (including exploratory experimentation and modeling) means and why our view is incompatible with 'unplanned data analyses'.

My 2 cents metatheorist.com/Is-the-Brain...

Just because I keep seeing the "replications show which effects are real" or "effects that didn't replicate once are flukes" heuristics repeatedly showing up on my TL today, I'm re-upping this thread. I beg everyone to please stop searching for quick and conclusive judgments on what's "true".

what an odd score-keeping theater. what do these criteria even mean? and ultimately it all comes down to statistical significance again. largest sample sizes will win science! have we learned anything?

Lots to unpack here….

Something interesting to read in between turkey and football

🌟🤖📝 **Boosting human competences with interpretable and explainable artificial intelligence** How can AI *boost* human decision-making instead of replacing it? We talk about this in our new paper. doi.org/10.1037/dec0... #AI #XAI #InterpretableAI #IAI #boosting #competences 🧵👇

Greetings my bluesky friends. I've been quiet for a while, but that is soon coming to an end. I've been getting recharged to post about science and all kinds of goodness.

To state the very obvious, this would put an avowed anti-vaxxer in charge of vaccines, allowing him to thwart the development and approval of life-saving shots. It's almost too horrific to fathom—a potential death sentence for our families and children. It's lethal. www.politico.com/news/2024/11...

Finally our editorial for the "What makes a good theory?" special issue is out (co-edited with @irisvanrooij.bsky.social, @jcskewes.bsky.social, Sashank Varma, and Todd Wareham). It's an interdisciplinary issue and the contents are transdisciplinary. #philsci #metasci #cogsci #ai #psych 🧪

Now forthcoming in Experimental Economics! papers.ssrn.com/sol3/papers.... This paper, with @yeli.bsky.social and Cade Massey, is a follow-up to my 2005 Management Science paper with Cade.

Metascience starter pack for the new folks! Interested in how science works? This is the pack for you! bsky.app/starter-pack...

What are your favourite women scientists in decision-making research and/or behavioral economics? Bonus points if their work is on decision-making under certainty.

Hey everyone, i created an addiction starter pack and I am sure I am missing folks - please send me a note to get added. go.bsky.app/KVDa5PD

Double descent enables a chat bot with a billion parameters to perform well and not overfit. But how does double descent work? I use simulations fitting linear regressions, plots, and tables for solving systems of equations to build intuition. medium.com/@jeanimal/ho...

I was a reviewer on this exciting paper by @fedeadolfi.bsky.social & colleagues that just came out in our forthcoming special issue on scientific theories. Finding worthwhile scientific problems in a theoretical space is a largely neglected part of the scientific process and deserves more attention!

Judgment and Decision Making (JDM) starter pack Happy to add you go.bsky.app/8nDKmwG

Article fresh off the print (no paywall)! The tl;dr is in the quoted thread. We advance a measure of replication distance to capture differences btw an original study & its replications. We also discuss how statistics needs to go beyond ideal assumptions to serve science better. #metasci #sts #stat