Accessibility is *absolutely* key but also hard because of the curse of knowledge. I've written down some writing advice here: www.the100.ci/2024/12/01/w.... If you're more of a technical person, consider teaming up with a substantive researcher for instant audience access.>

Explainers & tutorials are a great way to criticize current practices with a positive twist. Tailoring them to a specific substantive (!) subfield can greatly increase uptake. Forget about novelty; if some statistican said sth in the 70s but no one was around to hear it, say it again.>

After 3 years, this paper is finally out OA! We engage with the idea (that many entertain, if not endorse) that science can progress (or be fixed) thanks to its incentive structures alone - no explanatory role left to truth-seeking, curiosity, or scientific integrity. doi.org/10.1007/s112...

"The replication crisis in Psychology has undermined both intra-scientific and public trust. This review shows–using Wilholt's notion of epistemic trust–that replicability is fundamental for epistemic reliability" #metascience www.sciencedirect.com/science/arti...

Surprised to see this in PSE: www.sciencedirect.com/science/arti...

We had a very interesting visit and talk from Anna-Marie Ortloff at the lab yesterday, talking about how researchers in HCI and at CHI interpret effect sizes. Long story short: A lot of room - and need - for improvement. You can find her papers on this at: scholar.google.com/citations?us...

Job opportunity — Junior Professorship in Psychological Metascience @zpid.bsky.social leibniz-psychology.onlyfy.jobs/job/10kku5n7 h/t @bethclarke.bsky.social

Test severity may at once improve the ability of a replication study to reduce uncertainty, but it can also leave less uncertainty to be reduced to begin with.

Here is a thought I have not yet fully resolved: It seems to me that a paradox may arise when discussing replication value—specifically when weighing the feasibility of a replication study against the amount of uncertainty that the replication would reduce. ->

New 'Preliminary Report' submission format at Science and Medicine in Football. An idea that should be interesting for other journals! Explicitly make space for honestly reported smaller sample-size studies www.tandfonline.com/doi/full/10....

It seems, I was not up-to-date. Discipline specific examples have already been discussed, or should I say "will be discussed" ;) Your output is insane! @crist14n.bsky.social sportrxiv.org/index.php/se... bsky.app/profile/cspa...

The package formerly known as papercheck has changed its name to metacheck! We're checking more than just papers, with functions to assess OSF projects, github repos, and AsPredicted pre-registrations, with more being developed all the time. scienceverse.github.io/metacheck/

Just discovered this great preprint by @crist14n.bsky.social @lakens.bsky.social sportrxiv.org/index.php/se... I wish someone had given me tutorials/easy-to-understand explanations like these when I was doing my bachelor's degree!

Relatedly, I think SESOI is a tremendously useful and under-appreciated concept. It's become a regular tool in my power analysis workflows, and I wish I understood it sooner in my career.

These are exactly the right discussions to have IMO, especially when they help us better understand what knowledge we need (and might still be missing) for setting up meaningful, informative tests.

I think the thing that most people struggle with though is how to set that SESOI... @cspaeth.bsky.social and I been chatting about it in this thread (and I give examples of how we've gone about it... though our recent theory prediction + practical SESOI is best I think) bsky.app/profile/cspa...

There still seems to be a lot of confusion about significance testing in psych. No, p-values *don’t* become useless at large N. This flawed point also used to be framed as "too much power". But power isn't the problem – it's 1) unbalanced error rates and 2) the (lack of a) SESOI. 1/ >

New Commentary published in the German Journal for Sport and Exercise Research, emphasizing not only the need to specify a SESOI when testing a claim, but recommending also to start a discussion about best practices to determine a SESOI
link.springer.com/article/10.1...

My paper with @lakens.bsky.social and @annaveer.bsky.social - “Replication value as a function of citation impact and sample size” - has just been published in Meta psychology! open.lnu.se/index.php/me...

"the very core of science – an institution that promotes mutual criticism, while accepting our fallibility" bsky.app/profile/lake...

New release of PowerLMM.js! Browser-based power analysis for longitudinal models with dropout. Now includes: - Power analysis summary report - Reproducible & shareable configs (URL/JSON) - Calculations validated against R - Hypothesis region visualization powerlmmjs.rpsychologist.com

We're struggling to get reviews for our recent stage 1 at @pci-regreports.bsky.social www.medrxiv.org/content/10.1... If you've expertise in the methods proposed and/or subject area and would be happy to review then please get in contact and I can let the recommender handling the submission know.

I think this is kind of neat and I don't think anyone else has noticed it (I've looked and I can't find anyone who has) osf.io/preprints/so... Maybe I should back off "justification" language, but it's at least a remarkable coincidence. I still think someone else *must* have noticed it...

It also reminded me of the “stats maven” concept coined by @sanjaysrivastava.com some very long time ago, I believe here? thehardestscience.com/2014/12/04/s...

New study exploring a potential strategy for increasing affective responses during exercise while emphasizing the need to rigorously test related auxiliary assumptions and determine an SESOI for confirmatory research: doi.org/10.1080/0264...

When is it appropriate to deviate from a pre-registration, and how should this be done? 🧐 Join us to find out from Daniël Lakens @lakens.bsky.social at the next ReproducibiliTea! October 28, 1pm GMT. Sign up: www.eventbrite.co.uk/e/when-and-h... @tabeasch.bsky.social @reproducibilitea.org

Final version of our high-powered, pre-registered test of theoretically derived precise predictions regarding comparative RT interventions (full vs lengthened partial range of motion) in trained particiapnts now published in Journal of Sport Sciences: doi.org/10.1080/0264...

great blog post sometimesimwrongblog.wordpress.com/2018/06/14/n...

Doing the right thing, just because it is the right thing is underappreciated by academics.

"Each study has to be considered on its own. RCTs are fine, but they are just one of the techniques in the armory that one would use to try to discover things. Gold standard thinking is magical thinking." medium.com/@timothyogde...

why I’m happy to admit to my research failures, and you should too https://www.newsbeep.com/uk/173671/ The open-science movement has promoted the sharing of scientific protocols, statistical-analysis programs and data files. These efforts have…

Now that this is done, I have no goals left in life. meth.psychopen.eu/index.php/me... (* I hope I can repeat this joke a few more times.) #falsification #popper #science #replication #statistics #metascience

Sport and exercise science doesn't need papers telling researchers how to "make the most" of their small samples. The field needs to either grow the fuck up and collect sufficient data for adequate power/precision, or continue with their small samples but shut the fuck up and wait for a decent meta.

Not having to deviate from your preregistration is how you impress peers. You demonstrate you know something, because knowledge allows you to make good predictions. The more deviations, the less impressive the tests in a study are.

Me when speaking to sport and exercise science researchers who won't commit to a smallest effect size of interest.

I remember reading the preprint about a year ago... rereading with coffee this saturday morning. It reminded me, as it did when I first read it, why I find Lakatosian/Meehlian approaches that to an extent synthesise elements of both Popper and Mayo most appealing...

Too often, I see people talk about a replication as if the first study has established something, and the replication study is a double-check. What people often fail to understand is that we do not do replication studies to *check* a finding, but to *establish* a finding. 1/x

Second, beyond career implications, doing open science & especially replications often comes with immense emotional burden & affects mental health. Ongoing harsh criticism, suspicion, & hostility, from senior scholars with immense power in the field, isn't something people normally want or look for.

New Preprint: Rethinking Type S and M Errors. We argue Type S and M errors are not useful when designing or evaluating studies, and present alternative solutions to address the issues they were supposed to address (minimum effect tests, and correcting for bias with p-uniform). osf.io/preprints/ps...

New paper: Blaming the Thermometer for the Fever: Separating Misapplication from Method in Null Hypothesis Significance Testing, with @laabho.bsky.social and @jnwulff.bsky.social. We – strongly – push back on criticisms against NHST and argue for proper application. papers.ssrn.com/sol3/papers.... >

Exactly my first thought seeing d = 8: "No effect size shall exceed the preference for chocolate over poop"

If you're interested in watching me gibber about papercheck for an hour, I'm giving a local seminar tomorrow at 15:00 UK time. DM me before 14:30 and I'll send you the Zoom link. scienceverse.github.io/talks/2025-p...

There might not be 'a scientific method' but there is a very desirable property of scientific methods. If you are right, the procedure you use to make claims should tell you that you are right. And when you are wrong, the procedure you use to make claims should tell you that you are wrong.

Replication value as a function of citation impact and sample size: response to commentaries: https://osf.io/52nvj

Excellent opportunity for a PhD position with the wonderful supervisors Frank Renkewitz and Moritz Heene job-portal.lmu.de/jobposting/5... “Models, Measures, Moderators - Exploration and Explanation of Heterogeneity in Psychological Replications” - great topic for a PhD!

"Elitism in science has a legitimate basis, but its ubiquitous excesses affront the principle of universalism. Bad ideas get accepted because their proponents are members of the elite. More seriously, good ideas may be ignored ->

While individualism and isolated work remain common in #academia, coordination offers substantial benefits. This Comment urges systemic changes from all stakeholders toward more coordinated #science. @lakens.bsky.social @sajedehra.bsky.social www.nature.com/articles/s41...

The perverted scientific publication system explained for the German layman: www.youtube.com/watch?v=qtXA...

Appropriate sample sizes and a priori determination of them based on some justifiable SESOI are almost non-existent in our field... in some of our recent work we've really been trying to change that (sportrxiv.org/index.php/se...), but most treat n=10 and n=300 as the same level of evidence.

Maybe include one to Merton as well: