My Shiny app containing 3530 Open Science blog posts discussing the replication crisis is updated - you can now use the SEARCH box. I fixed it as my new PhD Julia wanted to know who had called open scientists 'Methodological Terrorists' :) shiny.ieis.tue.nl/open_science...

Oops. Ooooooooooooops. I do hope that nobody has been given or denied a job/promotion based on their SpringerNature citation counts in the past 15 years. arxiv.org/pdf/2511.01675 h/t @nathlarigaldie.bsky.social

This is why we have sample size justifications. Some people might mindlessly dismiss small N studies. But a well justified sample size (see online.ucpress.edu/collabra/art...) defends against such mindlessless.

New 'Preliminary Report' submission format at Science and Medicine in Football. An idea that should be interesting for other journals! Explicitly make space for honestly reported smaller sample-size studies www.tandfonline.com/doi/full/10....

@lakens.bsky.social pointed out on twitter that these text-mining studies (on mostly abstracts) are biased to overestimate the problem because exact test-statistics are more often reported for significant tests while non-significant tests are just described in words.

📚 Replication value as a function of citation impact and sample size: response to commentaries open.lnu.se/index.php/me...

Really enjoyed this! I esp loved the discussion of reverse p-hacking as a means of purposely generating null results. I could picture this happening more as null results become more acceptable--it'd be yet another way of creating a "clear story." Might I suggest calling it: "p-stacking"? 😉

Wouldn't it be great to gather top social science journal editors + experts on fraud-prevention to discuss better ways to fraud-proof our field @ the National Academies? This is happening! Step 1 is creating an organizing committee. Submit nominees by 11/7: www.nationalacademies.org/our-work/enh...

The package formerly known as papercheck has changed its name to metacheck! We're checking more than just papers, with functions to assess OSF projects, github repos, and AsPredicted pre-registrations, with more being developed all the time. scienceverse.github.io/metacheck/

Went on a trip to Paris for the day, giving a talk at Université Paris Cité, saying hi to former lab visitor Ethan Meimoun, and we even had time to pick up gluten free goodies for my wife at Copains on the way!

Just discovered this great preprint by @crist14n.bsky.social @lakens.bsky.social sportrxiv.org/index.php/se... I wish someone had given me tutorials/easy-to-understand explanations like these when I was doing my bachelor's degree!

Relatedly, I think SESOI is a tremendously useful and under-appreciated concept. It's become a regular tool in my power analysis workflows, and I wish I understood it sooner in my career.

Part 2 of our episode on p-hacking is out! Tune in to the latest Nullius In Verba episode here: nulliusinverba.podbean.com/e/p-hacking-...

Tenzing will also gently encourage best practices for Acknowledgments sections, by modeling them in its outputs. We are interested in talking to groups (like statistical consulting units or library professionals) who are frequently not acknowledged appropriately in papers about how this might help.

New paper finds that selective reporting remains the most replicable finding in science: journals.sagepub.com/doi/full/10.... I especially like their new exploratory metric 'p-values per participant'. Some papers had 11 p-values per participant! 🤯

Statistical tests (and thus p-values, if you use frequentist methods) are useful if you want to test a hypothesis or make a dichotomous claim (for a nice overview, see doi.org/10.1177/0959... by @uyguntunc.bsky.social et al.), regardless of whether N is small or large. >

5% is quite a lot if you think about it. Huge N gives you the luxury to reduce alpha by a lot and still keep very high power. E.g., alpha = 0.5% (0.005) would give you 98.8% power for the same effect (in a t-test). The best balance depends on the cost of each error type, see tinyurl.com/yut35b3u >

There still seems to be a lot of confusion about significance testing in psych. No, p-values *don’t* become useless at large N. This flawed point also used to be framed as "too much power". But power isn't the problem – it's 1) unbalanced error rates and 2) the (lack of a) SESOI. 1/ >

New paper finds that selective reporting remains the most replicable finding in science: journals.sagepub.com/doi/full/10.... I especially like their new exploratory metric 'p-values per participant'. Some papers had 11 p-values per participant! 🤯

Exciting new addition to the AsPredicted and ResearchBox research infrastructure toolbox: AsCollected ascollected.org A platform to document results provenance - where did data come from, who collected it, and who cleaned and analyzed it. So important!

New Commentary published in the German Journal for Sport and Exercise Research, emphasizing not only the need to specify a SESOI when testing a claim, but recommending also to start a discussion about best practices to determine a SESOI
link.springer.com/article/10.1...

Looking for help from the #psychology #metascience communities! This www.theoryfinder.com/theory-repos... online repository lists more than 200 theories (*), mostly from psychology. The authors' goal is to foster the use of theory ... I'd like some vetting of these theories. How do we do this?

Are you interested in thinking about which studies are worth replicating? Then you have 10 articles to dig into in Meta-Psychology, representing a very wide range of viewpoint on this topic, out now: open.lnu.se/index.php/me...

My paper with @lakens.bsky.social and @annaveer.bsky.social - “Replication value as a function of citation impact and sample size” - has just been published in Meta psychology! open.lnu.se/index.php/me...

In that case the whole SEVCO methods ontology project should be of interest to you. And this paper we just published, where we are using the SEVCO protocol to create a methods ontology for the GRADE Approach. It might give you some more ideas. www.sciencedirect.com/science/arti...

Given that many of the problems we have in research stem from appeals to authority, Daniel's blog post is a welcome reminder of other ways of solving the problem of proliferation of misused, misunderstood, and insufficiently valid methods.

"the very core of science – an institution that promotes mutual criticism, while accepting our fallibility" bsky.app/profile/lake...

New blog post: Why we should stop using statistical techniques that have not been adequately vetted by experts in psychology daniellakens.blogspot.com/2025/10/why-... where I reflect on how we should check the quality of novel statistical techniques.

Are you a journal editor in Psychology? There's a lot you can do to help support PCI Psych! PCI Psychology is a journal-independent peer review platform where journal editors like you evaluate and recommend preprints, that can then be published in PCI-friendly partner journals. 1/🧵 #PsychSciSky

Nature Psychology Review is a journal created by a commercial publisher because it is easy profit. Review articles are easy to write, get cited a lot. Slap Nature on it and you get a high IF. It should flip to a diamond open access journal. Submit your review articles elsewhere.

🚨 New publication: How to improve conceptual clarity in psychological science? Thrilled to see this article with @ruimata.bsky.social out. We discuss how LLMs can be leveraged to map, clarify, and generate psychological measures and constructs. Open access article: doi.org/10.1177/0963...

Nature Psychology Review is a journal created by a commercial publisher because it is easy profit. Review articles are easy to write, get cited a lot. Slap Nature on it and you get a high IF. It should flip to a diamond open access journal. Submit your review articles elsewhere.

True. It was not vetted by a real statistician, it was created by a real statistician, Jerry Brunner, Professor of Statistics at the University of Toronto, who also provided mathematical proof for the axioms underlying z-curve. replicationindex.com/2025/10/21/s...

Had a very nice drive with the wife listening to this excellent ep! I cant wait for the next one @lakens.bsky.social smritimehta.bsky.social!

We got our first review stating that as p-curve was recently criticized, and z-curve was also not 'vetted by a real statistician', the method should not be trusted. No statistical tool could ever damage science as much as this horrendously flawed authority argument.

I am writing a paper "Why small samples are all you need, small effects are really important, you do not need to correct for multiple comparisons, and invalid measures are useful'. But because I really believe it, and really not just for the attention and citations, I swear. 🙄

We got our first review stating that as p-curve was recently criticized, and z-curve was also not 'vetted by a real statistician', the method should not be trusted. No statistical tool could ever damage science as much as this horrendously flawed authority argument.

I am writing a paper "Why small samples are all you need, small effects are really important, you do not need to correct for multiple comparisons, and invalid measures are useful'. But because I really believe it, and really not just for the attention and citations, I swear. 🙄

Today I was happy to attend the excellent PhD defense of my good friend, and a wonderful scholar (and now Dr!) @mntunc.bsky.social

Dr Daniël Lakens presents 'When & How to Deviate From a Preregistration.' Essential for research integrity. Register: www.eventbrite.co.uk/e/when-and-h... #Preregistration #OpenScience #DataScience #Statistics #AcademicResearch @lakens.bsky.social

Today's email to an associate editor, declining to review for a big corporate publisher. This one focuses on Nature Publishing Group's practice of funneling rejected manuscripts to their newer journals with hefty APCs.

It was quite interesting – did wonder though what ecologists would have to say about it. My favorite presentation of the day though was by @ambra-prg.bsky.social on meta-analyses ... still thinking about her question of how big the problem should be for it to be a problem ...

Thank you so much for the invitation @lakens.bsky.social! It was a huge honor for me to give the keynote, and I enjoyed all the great talks and meeting everyone a lot!✨ #OpenScience @zpid.bsky.social

End of the Paul Meehl Graduate School Symposium, with wonderful chats over dinner. Last year was great, this year even better! So nice to talk to this crowd of metascientists. Thanks to @sajedehra.bsky.social for the organization of my favorite metascience event of the year!

Amazing few days hanging out in Rotterdam and Eindhoven for the Paul Meehl Graduate School Symposium... Thanks @sajedehra.bsky.social for organising such a great event, and @crist14n.bsky.social and @lakens.bsky.social for hosting me again. Met many lovely folks and chatted #philsci & #metascience.

Now Rasoul Norouzi from Tilburg University, talking about reproducibly automatically extracting causal statements from the scientific literature.

Now @ambra-prg.bsky.social showing in progress work on whether individual studies in a meta-analysis had a large enough sample size that the meta-analytic effect size could have been significant.

Very cool: Lisa Spitzer has made a deviation from preregistrations template, which you can find here: apps.leibniz-psychology.org/prp-dev/

Second keynote of the day, Lisa Spitzer talking about how to foster preregistration, with a lot of insights into how it was to be a metascience PhD researcher!

Cas Goosen is presenting the idea of multi-stage preregistration, a way to improve the evaluations of the severity of deviations (which are common).