Are you interested in Shaping an Open Science Culture? Apply for this 2 year post-doc position at UMC Amsterdam. Joeri Tijdink is the PI, but you will work in a large interdisciplinary team (including me). werkenbij.amsterdamumc.org/en/vacatures...

Still a few days to apply!

Bad idea for many reasons, not least that LLM-based workflows for this type can produce wildly different results depending on even just a few analytic decisions with many plausible choices: arxiv.org/abs/2509.13397

1. "From a patient experience perspective, therefore, the most salient information from an RCT is the trajectory of change within the antidepressant arm. " I disagreed with this claim when I reviewed this paper. It was now published in a different journal. journals.sagepub.com/doi/full/10....

From now on, I'm going to write papers where I take an arbitrary performance metric, restrict the sample to top performers on that metric, and then hunt for negative correlations with everything else in the truncated sample. I'm at a business school, so this should be extremely lucrative.

Some research fields have a type of paper that basically asks: "What if a more expensive but less reliable machine can do [this] instead?"

Following up on our discussions in the last days on implausible effect sizes: psilocybin RCT with n=40 and an odds ration of 18. jamanetwork.com/journals/jam...

Sleuths face various legal risks but they can also use the law for their advantage. ⚖️ COSIG's newest guide, number 3️⃣5️⃣, covers legal considerations for taking part in forensic peer review. Read it here: osf.io/q3y7s

It's now 4½ years since I first thought of writing it, and 3½ years since I actually started writing it, but today I finally got around to submitting the preprint. Say hello to SPIV. 🧵

If you've tried using INSPECT-SR to assess non-randomised studies and would be up for having a quick chat about your experience using it, let me know!

@taymalsalti.bsky.social has done trojan work trying to reproduce the results of three meta-analyses, finding the usual host of issues of effect sizes that are apparently incorrect or non-reproducible Our article is out now: doi.org/10.1111/ejn....

Eagle-eyed @grinschglsandra.bsky.social spotted this mess of a plot, with bars that don't match percentages. We, with @malte.the100.ci, quantify a host of other issues we spotted here. pubpeer.com/publications...

“If researchers observe Cohen’s d = 4, no they didn’t.” Wolfgang and I have an idea for how to flag errors that produce erroneous large effect sizes during analysis, when they can be corrected. Feedback welcome!

Ian Hussey (@ianhussey.mmmdata.io) contacted me about the recurring problem of unusually large SMD values appearing in published meta-analyses. Very large SMDs can result from using SEs in place of SDs in their calculation, which is a mistake I have also seen in practice. 1/4

Great to hear a bit of clear science communication from @marcuscrede.bsky.social in the latest @ifbookspod.bsky.social. It's so wild that 0.5% of observed variance can be spun into TED talks, policy impact, and airport books. 🤔🤯😭 podcasts.apple.com/gb/podcast/i...

Does anyone know of a tidy dataset with a large number of common multi-item self-report psychology scales? Ideally clinical. The data does not have to be within-subject, just a variety of common scales with decent N. I have: - Bainbridge et al - AIID dataset - Ideologies 2.0 dataset

Can accommodate some more people at this Introduction to INSPECT-SR workshop tomorrow (April 30th). Book here or email/ message me directly: www.trybooking.com/uk/GATM It says (N America) because the time was chosen for that region, but all welcome

Reminder: when authors observe Cohen's d = 4, no they didn't. Article now retracted: www.nature.com/articles/s41... Critique: osf.io/preprints/ps... Blog on plausibility of effect sizes: trustworthy.scientific.claims/posts/if-res...

Analysis of 19k ads of authorship for sale provides unprecedented insights into papermills. Great work all involved.

I should really not have a discretionary budget

Good news everyone 🥳 Our (w @vincentab.bsky.social) primer on models as prediction machines (with the marginaleffects package) is finally officially published!> journals.sagepub.com/doi/10.1177/...

Don't be shy to take on a little two-week side project. These five months will be the most precious three years of your academic journey.

1/ "Silicon samples" are becoming more and more common in research and polling. One problem: depending on the analytic decisions made, you can basically get these samples to show any effect you want. The updated version of this preprint is now online! THREAD🧵 arxiv.org/abs/2509.13397

Reminder: If researchers find Cohen's d = 6, no they didn't. trustworthy.scientific.claims/posts/if-res...

1. The paper with the implausibly large effects of Omega-3 fatty acids on mental health was now retracted. A little thread on the process where @ianhussey.mmmdata.io and I was involved. www.sciencedirect.com/science/arti...

I’m hiring a PhD student! The candidate will work alongside @zefreeman.bsky.social, who is joining our research group as postdoc. jobs.unibe.ch/job-vacancie...

Think we have people from every continent registered to attend the next INSPECT-SR training workshop. Join them! Times shown are UK - Thursday 30th April 6pm (see next for May date) www.trybooking.com/uk/GATM

Congratulations to @jdwilko.bsky.social for the citation of INSPECT-SR in testimony to the US House science oversight subcommittee and its mainstreaming of the scientific value of sleuthing.

We are inviting applications for a two-year postdoctoral position in a collaborative meta-science project on the effectiveness of data and code sharing policies in research-performing organizations. www.tue.nl/en/working-a...

How every layer of science's "self-correcting machinery" failed when Iva Veseli and I simply wanted to reproduce the findings of a high-profile study on gut microbiome and autism: merenlab.org/2026/04/15/u...

Many folk are surprised to discover thay Risk of Bias assessment tools tend not to interrogate the question “Did this study actually happen? And are its results trustworthy enough to believe?” Jack’s Cochrane endorsed INSPECT-SR checks have done a lot to mainstream such Trustworthiness Assessment.

Very nice INSPECT-SR website containing the guidance and editable template, made by @ianhussey.mmmdata.io inspect.sr

Time for this again

Fun facts - The original study reported 23 IQ point gains - Jordan Peterson tweeted about it (sort of) - @jamiecummins.bsky.social new preregistered RCT shows null effects

A failure to find effects of relational operant training on scholastic aptitude of school children: A randomised controlled trial: https://osf.io/cauvz

Very pleasantly surprised to see my recent preprint being discussed in this NYT article on silicon sampling and its implications for opinion polling. More forthcoming work on this coming soon! (H/T @rohanalexander.bsky.social) www.nytimes.com/2026/04/06/o...

Don't miss out on new replications with the Zotero Replication Checker! This is the current set of >1.6K studies in the FORRT Library of Replication Attempts (#FLoRA). Many of them belong to large-scale projects and are not even cited in the final report, so how to keep track of them?

Majestic, well done @rickcarlsson.bsky.social

Our research group in Bern has been awarded two @snsf.ch project grants! We will hire a number of new postdocs and PhD students. Congratulations to @jamiecummins.bsky.social @malte.the100.ci and @ndevito1.bsky.social whose project deepens the links between @bennettoxford.bsky.social and @unibe.ch

Happy "large enough n" day

I just read a paper proposing that we use heart rate variability measured from developers as they're coding to create a detection flag for code quality issues. This is so theoretically and practically preposterous I have to ask myself if such things are a joke

Shocking to think that telling people to imagine eating ice cream is not the same as having them eat ice cream. Don't get me wrong, it's good someone is doing these studies, but the wild claims based on vignettes never cease to amaze me.

Do conjoint/factorial/vignette experiments reflect choices in the real world? Are hypothetical scenarios in the artificial survey context externally valid? Do hypothetical bias, intention-behavior gap and social desirability biases undermine validity? Two cautionary studies on this question:

3 new studies on using psychedelics as treatments for mental disorders just came out; 2 in JAMA Psychiatry, 1 in Lancet Psychiatry. I reviewed one of these submissions. Here are the core results. 🧵

New report: Joel, Eastwick, & Finkel (2017) “Is Romantic Desire Predictable? Machine Learning Applied to Initial Romantic Attraction”. Based on the review by @florianpargent.bsky.social, we find Minor Errors that do not affect the core conclusions of the manuscript. osf.io/hxnum

It's turning into some sort of meme in the meta-science discourse that somebody suggests "practice X is a reasonable default" to which then somebody else responds "oh so you are arguing there should be no exceptions" 😭

9am this morning (UK-time). Introducing INSPECT-SR: a tool for assessing trustworthiness of RCTs: www.cochrane.org/events/inspe...

It was an honour to have RegCheck featured in this most recent issue of REACH magazine. Check it out below!

Meanwhile my public health colleagues