🔓 It's great to see authors sharing their experiences with publishing on MetaROR (MetaResearch Open Review) — our open review platform for metascience using the publish–review–curate model: www.openscience.nl/en/cases/the...

We are about a month away from releasing a complete refresh of the OSF user interface. The team has been working on this for a very long time, and we are very excited to be able to share it soon. A preview picture:

Many interesting tidbits in @simine.com's talk. A selection: - Journal prestige depends on factors like aims and scope, selectivity, and impact factor, but changes in these factors do not always lead to changes in journal prestige - journal prestige is sticky #PRC10

eLife (talk by Nicola Adamson) uses a publish-review-curate method and uses common terms to assess manuscripts. For strength of evidence: exceptional, compelling, convincing, solid, incomplete, & inadequate For significance of findings: landmark, fundamental, important, valuable, & useful #PRC10

New peer review dataset incoming! Involves authors, topic area, editorial decision, author characteristics (institutional prestige, region, gender), BoRE evaluations, review characteristics (length, sentiment, z-score, reviewer gender). (Talk by Aaron Clauset) #PRC10

Christos Kotanidis checked differences in abstracts between submissions and published papers & assessed whether these differences indicated higher or lower research quality. Abstracts typically improved, especially in big five medical journals. Evidence for the effectiveness of peer review? #PRC10

Andrea Corvillon on distributed vs. panel peer review at the ALMA Observatory: Most experienced PIs no longer have the best ranks in a distributed review system, but why that is remains unclear. #PRC10

Interesting to see that the conference review process (and publishing norms) are do different in the field of computer science compared to other fields. How do these differences come about? Fundamental differences between fields or chance and inertia? #PRC10

Di Girolamo explains why the use of the phrase "to our knowledge" lacks reproducibility and accountability. Good trigger to make an edit in a grant proposal I'm writing. #PRC10

Data repurposing: taking existing data and reusing it for a different purpose. (Presentation by Yulin Yu) Studies repurposing data are at higher risk of bias, so make sure to preregister them (check here for a template): research.tilburguniversity.edu/en/publicati... #PRC10

Ian Bulovic used OpenAI's GPT to assess selective outcome reporting. Findings: - Much outcome switching but decrease over time - Industry-sponsored trials most at risk - Assessing outcome switching may seem trivial but is even hard for human coders #PRC10

A meta-perspective by Malcolm Macleod on the presentations at #PRC10. Are we going for low hanging fruit too much in research on peer review / publication?

A question to kickstart day 2 of #PRC10: How would you measure the quality of peer reviews in a scientific study? Single question? Scale? How many raters? AI?

Leslie McIntosh: Markers of (dis)trust in science: Pay attention to email addresses (use of hotmail.com and underscores) and institutional affiliations (new and unknown organizations without verifiable addresses) #PRC10

John Ioannidis: "We need more RCTs" I agree, so here is an urgent call to the representatives of journals at #PRC10: Let's empirically test suggested improvements to peer review like open reports, open identities, structured review, results-free review, collaborative review, etc. Get in touch!

How do paper mills operate? I always thought they were in cahoots with illegitimate journals but apparently they target normal journals as well, with the editors of those journals playing no role. (Talk by Tim Kersjes from Springer Nature) Could open review reports solve this issue? #PRC10

@royperlis.bsky.social based on a study using Pangram to detect AI use in papers: "Less than 25% of authors using GenAI are disclosing its use" Why is this the case? Do people feel shame for using AI to improve their studies / papers? Do journals discourage (disclosure of) AI use? #PRC10

Start of the talk by @mariomalicki.bsky.social: "Only 0.1% of journals provide some sort of open peer review" Shockingly low number for something that should be standard given that reviews are part of the scientific discourse and therefore should be public. #PRC10

What do researchers use AI for? Talk by Isamme Al Fayyad on the first study from his PhD @maastrichtu.bsky.social #PRC10

TIL: In biomedicine there is a thing called "medical writers" who are hired to prepare papers for publication, and write regulatory documents and grant proposals. Questions: Why aren't the researchers writing papers themselves? Should medical writers be authors on the resulting papers? #PRC10

We found that the results of 42% of registered studies on the OSF are never publicly shared journals.sagepub.com/doi/10.1177/.... Most common reasons are null results and bad logistics. Plan your studies carefully (equivalence testing and good time management) to prevent research waste!

Simulation studies are widely used, but have historically lacked clear guidance around preregistration. In response to this gap, a team of researchers have developed a new template for preregistering simulation studies, now available on OSF. 🚀 Read our Q&A: www.cos.io/blog/intr...

New blog post: Easily download files from the Open Science Framework with Papercheck daniellakens.blogspot.com/2025/07/easi... Downloading all files in an OSF project can be a hassle. Not anymore, with Papercheck! Just run: osf_file_download("6nt4v")

When MetaROR launched at the end of last year, we published four articles. One of these has now returned to our platform for another round of peer review. 👇 Read the editorial assessment, reviews & full manuscript text at MetaROR (incl. links to the first version and first round of reviews) [1/3]

Non-profit publishers (PLoS, Oxford, Cambridge) and publishers partnering a lot with academia (Wiley) are much more committed to open science than the others - MDPI being the worst by large. (4/5)

Registration now open for this big meta-science unconference in Berlin, hosted by the QUEST Center for Responsible Research! Take your pick from 10 projects to collaboratively work on a tangible output that will make science better: rr-in-action2025.org/programme/ #metascience2025

Shape research culture + practices! Join the Responsible Research in Action Unconference 2025, Sept 22–24, Berlin, to collaborate on innovative solutions – from reproducibility checks to researcher-driven funding models. rr-in-action2025.org @dfg.de @ucoimbra.bsky.social @berlinualliance.bsky.social

Be part of the future and submit your metascience work to MetaROR! #metascience2025

We had a great time at our session at #metascience2025 on Monday! We got a lot of new members and connections that joined our community over the last days. Missed our talk, and want to join PYMS or learn more? Find one of our board members at the last day of the conference!

@metaror.bsky.social is a scholar-controlled peer review service for metascience research. Check us out! Metaror.org #metascience2025 (I am an associate editor at MetaRoR) @ludowaltman.bsky.social @aidybarnett.bsky.social

Hot topic at the moment: AI in peer review. Ashia Livaudais from startup SymbyAI discussed this during a session on peer review at #metascience2025 Conclusion 1: the reviews of human-AI combinations were rated higher in quality than reviews of humans or AI by themselves

PYMS has a new board and it is booming! Don't miss this session in London if you are an early career meta-scientist (or are interested in meta-science careers)!

We are looking forward to Metascience 2025 in London next week! Especially to the panel where we will discuss PYMS! Come discuss career paths with us Monday 14:00 - 15:30: Session 2, Panel T2.3; Snakes and ladders: mapping research careers

What if you want to deviate from you preregistration but keep the label "confirmatory"? Cas Goos introduces multi-stage registration as a solution. Read all about it in the semi-new (I somehow missed a month of blogging?) blog post from the Meta-Research Center: metaresearch.nl/blog/2025/5/...

📣New Preprint Alert! Esther Maassen meticulously simulated the effect of p-hacking and publication bias on effect size & heterogeneity estimates. 💡bad: selective outcome reporting & optional dropping 💡bad: publication bias 💡not so bad: optional stopping/outlier removal osf.io/preprints/ps...

We are reviewing the current state of scientific publishing, and we want to hear from researchers on their perceptions of publishing in the current context and how it might develop in future. Complete our short survey now:

Program finalized! We're very excited to announce our program for the PYMS Summer Symposium 2025. Check out the abstracts here: tinyurl.com/muaa5dbd Don't want to miss out? You can still sign up until the end of this week (23rd of May): forms.gle/yvgVBf8VKCFu...

Out now: our paper on how to deidentify data sets in the social sciences while navigating privacy concerns. The paper includes a step-by-step de-identification guide and two worked examples, with tons of accompanying material on OSF. journals.sagepub.com/doi/10.1177/...

With all due respect to Cathleen's excellent reporting (must read imo), I'd say the thing being most exploited here is our failure to insist on open, expert peer-review, particularly from journals that claim to be peer-reviewed.

The PCI Psychology website is live! psych.peercommunityin.org We're working hard to be ready for launch (planned May 2025) but there's plenty you can do in the meantime to help it along. Go check it out! #psychscisky #scipub

We are actively looking for reviewers and recommenders for @psych.peercommunityin.org ! Please consider contributing to an alternative community-based model for peer review: psych.peercommunityin.org/about/about

🚨 New PhD position at CWTS! We're excited to announce a fully-funded PhD position on a crucial topic: Paper Mills and Systematic Manipulation in Research Publishing 📄 www.universiteitleiden.nl/en/news/2025...

📣 COS is excited to launch the Replicability Project: Health Behavior (RPHB)! Seeking researchers with expertise in quantitative methods in health behavior, public health, epidemiology, health policy, & related fields for this large-scale, multi-team replication effort. ➡️ Details: www.cos.io/rphb

🚀 Registrations & submissions are open for the PYMS Summer Symposium 2025! Join us June 5th at Tilburg University for talks, a hackathon, dinner, and networking. Travel funds available! Deadline: April 30th! Sign up: forms.gle/wABucttrY8Ur...

I'm getting the chills from this invite 🥶🥶🥶

Very good start of the new @metaror.bsky.social journal, as the public peer reviews are as critical of @jamesheathers.bsky.social recent claim that 1 in 7 papers are fraud as I want my peer metascientists to be of such badly supported bold claims. Read the reviews here: metaror.org/kotahi/artic...

We (AIMOS and RORI) launched a metascience platform, MetaROR.org, several months ago 😀. Submit your metascience-related preprints and get peer reviews! André Brasil articulates the vision here, reflecting the hard work he's done to get the platform working for us. katinamagazine.org/content/arti...

Next week, the QUEST Center for Responsible Research in Berlin is organizing a week of free (!) tutorials about all kinds of scientific methods. I'll feature with a session on Friday about how to best preregister your research. More info: bihealth.org/de/aktuell/l...

PYMS is back! Proud to have been part of the board together with @annemscheel.bsky.social, Noah van Dongen, and Sarahanne Field, but happy to pass the baton to Cas Goos, Sajedeh Rasti, @raphaelmerz.bsky.social, and @anoukbouma.bsky.social. Exciting things ahead!

📣 From today, all #ClinicalTrials in the #EU are governed by the Clinical Trials Regulation. The Clinical Trials Information System (#CTIS) now supports submission, assessment and oversight of all trials in the EU. Read more: www.ema.europa.eu/en/news/clin...