Great to see that the future for Regcheck is bright. The tool uses LLMs to compare PAP with published study

WE ARE HIRING! Come work with us (me + @malte.the100.ci) on developing and validating RegCheck. PhD or postdoc applications welcome. Folk with backgrounds in meta-science, psychology, (pre)clinical trials, and/or economics are especially invited to apply. jobs.unibe.ch/job-vacancie...

Beth continues to kick ass. So glad this kind of permanent academic position exists in metascience, and so glad @bethclarke.bsky.social is paving the way. Hopefully there will be lots more departments adding metasciencey people like her in the years to come! A pretty smart move if you ask me.

Very excited to announce that I’ll be joining @zpid.bsky.social in Trier, Germany as a tenure-track Jun.-Prof. in Psychological Metascience in November! 🤸🏻‍♀️✨

Congratulations, Professor Clarke! What an outstanding achievement. Seeing you leave is a loss for our group, but seeing you get a tenure track position is a huge win for meta-science, and, I dare say, psychology. Congratulations again to you, and @zpid.bsky.social!

Our metacheck package now has an online manual! In 29 short chapters we explain how to set up metacheck (running all modules 100% locally if you want), and explain the 18 available modules. www.scienceverse.org/metacheck_bo... We are still in beta, and would love to get feedback!

I have managed to abstract the template for replies to critiques.

🚀 During his six-month research visit at the Bennett Institute, @jamiecummins.bsky.social has worked on developing and evaluating AI-assisted tools to improve research transparency, reproducibility, and trustworthiness. Read his reflection (link in comments 👇)

I was very fortunate to work as a visiting scholar at the Bennett Institute at University of Oxford for 6 months. Below I reflect on that experience, what we achieved, and what's yet to come! It's a privilege to get to continue to work with these great folk. www.bennett.ox.ac.uk/blog/2026/06...

I checked the prereg they linked with the tool @jamiecummins.bsky.social built...they preregistered a bunch of stuff they never reported... anxiety and depression scores for example. I really wonder how they even placed this in a medical journal, there's nothing medical in there.

One of the perks of playing music in a wedding band is getting used to performing things with very little prep time. When you have played someone's first dance after learning the song 15 minutes before, doing talks on the fly suddenly feels a little easier.

Our lab loved this paper when it came out as it combined two of our interests, technology effects and schmeasurement. Plus: Registered report, open data. We read it for our journal club and found issues in the Slovak sample that we now posted to PubPeer: pubpeer.com/publications...

The paper "The Bots Ruining Social Science Are Not Bots at All" by Jaffe et al. got a lot of attention when it came out in Perspectives on Psychological Science. We read it in our journal club and have a few comments that we now posted to PubPeer pubpeer.com/publications...

I built a website that tracks open source tools that provide automated checks for scientific manuscripts: lakens.github.io/automated_re... It provides a summary through RSS feed that you can subscribe to. It tracks 13 repositories - if you want your repo to be included, let me know!

My PhD Sajedeh Rasti has published a World View piece in Nature today: "Don’t compete, collaborate: why collective funding applications are the future" www.nature.com/articles/d41... She discusses how she successfully brought together a team of meta-scientists to collaboratively apply for funding.

🧠📑 The Bernese tool #RegCheck uses AI to systematically compare clinical trial registrations with published articles. This makes #OutcomeSwitching and #HARKing visible and strengthens the evidence base for reliable systematic reviews 👇 ▶️ sohub.io/gl4r @jamiecummins.bsky.social

We launched our beta version 0.0.1 of Metacheck today! After our second year of work on the tool, we now have validated modules, hugely expanded functionality, optional (or local) LLM use, checks inside code files, and much more! Try it out in R now: www.scienceverse.org/metacheck/

And some words of warning by @jamiecummins.bsky.social

The Bennett Institute is a special place that blends research with scientific infrastructure development in a way that we all aspire to but rarely achieve. Consider applying to work there!

📣 We’re hiring! ⚙️ Head of Engineering (closes 8 June) 👩‍💻 Senior Developer (closes 24 June) 💻 Staff Developer (closes 24 June) If you'd like to use your software skills to improve healthcare and medical research, the Bennett Institute might be the place for you! 🔗 www.bennett.ox.ac.uk/vacancies/

Paying peer reviewers works. Expanded Fast & Fair experiment @biologyopen.bsky.social: • 5.5 vs 37.7 working days to decision with reviews • ~3 vs ~9 reviewer invitations per manuscript • no reduction in editor-assessed review quality • similar acceptance rates www.biorxiv.org/content/10.6...

Did someone say “explanatory fiction”?

Guys, how do you invent a vaccine? Or even wilder, how do you invent a vaccine during your PhD?! In a new episode of Hard Drugs, we talked to someone who did just that: @kat-a-collins.bsky.social! Plus: • Why did the first malaria vaccine take so long? • Could we make better vaccines today?

I will be speaking at this super interesting conference - sign-up and join us in Eindhoven!

New blog post: Evaluating Dr. Cuddy’s Claim that the Debunking of Power Posing is a Myth. daniellakens.blogspot.com/2026/05/eval... On an AI generated description of a non-existent study, incorrectly citing findings from studies, and the importance of scientific criticism.

1. Here's a summary of a preprint with Alison Avenell, just published on MedRxiv on sharing of individual patient data (IPD) in clinical trials. medrxiv.org/cgi/content/... (NB We will revise title and abstract as our focus was more on rates of IPD-sharing than adherence to data-sharing policies)

@jackstripling.bsky.social interviews Dan Ariely about his years-long correspondence with Jeffrey Epstein www.chronicle.com/podcast/coll...

Austin et al. (2026) advocated for AI in hypothesis-driven scale validation research. In this preprint, led by @wchiggins.bsky.social, we (@malte.the100.ci @bethclarke.bsky.social) sketch out several weaknesses in this paper, and offer recommendations for avoiding them. osf.io/preprints/ps... 1/🧵

Austin et al. (2026) advocated for AI in hypothesis-driven scale validation research. In this preprint, led by @wchiggins.bsky.social, we (@malte.the100.ci @bethclarke.bsky.social) sketch out several weaknesses in this paper, and offer recommendations for avoiding them. osf.io/preprints/ps... 1/🧵

YESSSSSSSSSSSSSSSSSSSSS COYG!!!!

The Arsenal. Your Premier League champions.

Collabra: Psychology, the official journal of the Society for the Improvement of Psychological Science, seeks applications for Editor-in-Chief! The SIPS Publications Committee encourages editors with innovative ideas about how Collabra: Psychology can best accomplish its mission to apply.

We actually didn't discuss this paper in our cakeclub, it was more chance that I came across it and it didn't take long until we (with @ianhussey.mmmdata.io @malte.the100.ci @jamiecummins.bsky.social) discovered different issues.

There's an interesting conversation unfolding on PubPeer. Inconsistencies were flagged in a paper by a "global keynote speaker on AI&neuroscience" and then the author casually mentions on LinkedIn that she also has pre-post-fMRI for 300 people 👀> pubpeer.com/publications...

Recommendations for incorporating LLMs into psychological research: A Commentary on Austin and colleagues (2026): https://osf.io/yr6d3

This paper has now been accepted at Advances in Methods and Practices in Psychological Science!

NEW: You can now get a Ph.D. in antisemitism studies. Should you be able to? I wrote about the rise of antisemitism studies in the academy post-October 7th: a burgeoning field flush with cash, politically in vogue, and rife with infighting. www.chronicle.com/article/the-...

Are you interested in Shaping an Open Science Culture? Apply for this 2 year post-doc position at UMC Amsterdam. Joeri Tijdink is the PI, but you will work in a large interdisciplinary team (including me). werkenbij.amsterdamumc.org/en/vacatures...

Bad idea for many reasons, not least that LLM-based workflows for this type can produce wildly different results depending on even just a few analytic decisions with many plausible choices: arxiv.org/abs/2509.13397

There is a subgenre of this problem among psychology researchers in particular that goes: 1. Claim we don’t understand LLMs 2. Get LLM to complete psychological task 3. LLM shows some effect 4. Claim that this provides some kind of understanding about LLM mechanisms.

Preprinted! Been working on this for ~1 year w wonderful coauthors Nancy & Mark: > Understanding Artificial Neural Networks: Mysterianism about Known Mechanism is Mysticism doi.org/10.5281/zeno... Have you noticed many experts say they don't understand AI? What's going on? 🤯 Let us tell you... 1/

It's now 4½ years since I first thought of writing it, and 3½ years since I actually started writing it, but today I finally got around to submitting the preprint. Say hello to SPIV. 🧵

Some research fields have a type of paper that basically asks: "What if a more expensive but less reliable machine can do [this] instead?"

I’m here!

If you've tried using INSPECT-SR to assess non-randomised studies and would be up for having a quick chat about your experience using it, let me know!

As someone researching cognitive offloading and human–AI interaction, methodological clarity and reproducibility in this area are especially important to me. Glad we were able to document our concerns on @pubpeer.com

Eagle-eyed @grinschglsandra.bsky.social spotted this mess of a plot, with bars that don't match percentages. We, with @malte.the100.ci, quantify a host of other issues we spotted here. pubpeer.com/publications...