We can no longer trust that survey responses are coming from real people.”

There’s no ‘could’ about doing less research. Colleagues here being told they can’t go for bids that don’t cover overhead (ie all the charity funders)

Really enjoyed this episode of @bjkspod.bsky.social with @mbaldwin.bsky.social talking about the history (and possible future) of peer review: www.buzzsprout.com/1390924/epis... Interesting to learn just how recent the idea that "research must be peer reviewed" is (70's), and that its original...

How can we reform science? I have some ideas. But I am not sure you’ll like them, because they don’t promise much. elevanth.org/blog/2025/07...

Why I'm more optimistic than others about metascience (or metaresearch) (I wanted to write up some thoughts on #metascience2025 but I just don't have the time so this thread will have to do.)

This is an interesting reflection on the relationship between metascience and research culture. There’s some excellent and relevant work that gets done under the banner of “research culture”, especially around things like inclusivity, recognition, and community building and I agree...

as spectator and also participant in various metascience-informed things like prereg, github etc, I can't help but notice a parallel between metascience-as-management (rather than science) and academic-leadership-as-management (rather than academic leadership)

If a drug company funds a study showing that their product is highly effective, we would be rightly skeptical of the results; if tech companies fund #metascience into how AI can improve science should we be equally sceptical?

Chatbots — LLMs — do not know facts and are not designed to be able to accurately answer factual questions. They are designed to find and mimic patterns of words, probabilistically. When they’re “right” it’s because correct things are often written down, so those patterns are frequent. That’s all.

Looking forward to being part of the panel at this online event this afternoon on critical Metascience, I'll be talking about the alignment between open research reforms and academic capitalism. Details in the post below 👇 #metascience

A sunny Friday afternoon so I'm in a beer garden working my way through my stack of "interesting papers to read" and thought I'd do a mini thread of the best. Today's paper (actually a book chapter) is Jochen Gläser's 2019 "how can governance change research content?" doi.org/10.4337/9781...

This is a great post and really underscores the need for metascience reforms to seriously consider the context of their implementation, and the impact this has on researchers - "Get to know your audience"

Let's have some fun. In this NYT story, we see a picture of an OpenAI billboard on a college campus, suggesting the following prompt: "Give me a guide for mastering this Calc 101 syllabus. In two weeks. Small steps please." What happens when we actually use ChatGPT to do this? THREAD!

This is a great post and was nice to see my paper on the invisible workload of open research quoted (journal.trialanderror.org/pub/the-invi...). Imposing mandates on researchers to perform time intensive open research practices is particularly unreasonable when combined with cuts to time and budgets

🆕 COS statement on the "Restoring Gold Standard Science" Executive Order The EO names important open science principles, but their application here is counter to open science's purpose to accelerate discovery, advance treatments, & create knowledge. 📄 Read our statement: www.cos.io/about/news/c...

Just astonishing: which airport book is ‘email jobs’ from, and which 1970s fantasy about everyone being in ‘production’ is this? Just utter ‘know your place’ stuff.

Its just warm words - like all funding agencies, universities, journals ("excellence" etc)... Only the implementation & detail really matters to me. (Not taking 'a side' on this one!)

Here's a thought experiment for you all: If a seemingly sane and science-literate government (I.e., not Trump) passed a law saying that all government policy must be based on reproducible, transparent research thats open to null findings, how would everyone react? #metascience

Any researchers in the Netherlands interested in the effects of #openresearch policies/practices on academic workload or administrative burden? Please get in touch! I would love to collaborate with you on a project for this funding opportunity

There’s the weaponization of open science y’all. Basically ripped from @cos.io’s website. The ROI for leaders speaking up is the highest it will be now, and only decline over time as these policies take shape and grow teeth.

Trying to keep up with this #metascience EO debate but this seems bang-on. To everyone saying "if it wasn't the science reform mvt it would be something else to justify this"...what else could possibly exist that would support the EO so effectively than a decade of 'science in crisis' literature?

1. It's Fustilarian Friday and today's drop is the "Restoring Gold Standard Science" executive order. My read is that this is designed to (1) protect far-fringe figures such as the antvaxxers now heading our health agencies, and (2) kneecap efforts to use scientific evidence in regulatory policy.

Also the anti-pluralist stance embedded in the reform to further this reductionist narrative has from the getgo had authoritative, at times fascist tendencies. This isn't the time to talk about our good intentions; we're way past that. It's time to take responsibility, to be humble and accountable.

Again, this is not a blame issue for me but points to the fact that (as I have said many times) the seemingly apolitical nature of science reform works against the (imo) real issues of labour and collective organisation.

This paper is an interesting case study example to come along so soon after the equally viral discussion on "stop using the phrase 'scientific consensus' and start using 'convergent evidence' instead'" - see www.science.org/doi/10.1126/...

Hello followers! I'm back on Bluesky after 6 months of work-free parental leave. What have I missed? Please share with me any interesting Open research/Metascience publications, blog posts, or juicy scandals I should catch up on! #openresearch #metasci

Very honoured to have written a little something for @aeon.co about zero, absence, and consciousness. It was a real pleasure to write, and very curious to know what people think! 0️⃣🧠 #PsychSciSky #neuroskyence #consci

If you bookmarked "Rankings are all bullsh*t anyway, why not do my own?" but did not get to read it yet, worry not, we've got you covered with a short blog post about it Bonus track: there is a video by one of our vloggers in it 🤩

this is a great piece, and I just want to propose that the *funniest* research development would be for all of the researchers involved to quit Posting at each other for 2-3 weeks, self report the effects on their mental health, and coauthor the paper

Lots of new followers (hi!) so thought I would do an overdue thread on my recent paper “Open Research Reforms and the Capitalist University: Areas of Opposition and Alignment” (doi.org/10.1525/coll...) #metasci

Hang on a minute…

The results LOOK pretty replicable? That's not how any of this works and you'd think someone who runs a reproducibility project would know that damn well. Why would Nature close an article with so foolish a comment? www.nature.com/articles/d41...

Some great points here. The proposals will just shift the focus of admin to grant capture 1. Grants become only source of research £ for unis 2. Grants are in many (most?) cases lotteries 3. Buy as many tickets as possible All REF staff and administration redeployed to increase grant applications

This week, the touchstone paper purporting to show that there are replicability benefits of preregistration was retracted. For me this highlights a long-running disagreement I've had with core CoS strategy. I like the idea of making things possible, easy, and rewarding.

This is fantastic persistence from Joe to keep the story around this retraction open and keep pressure on the authors to be transparent about the changes to the manuscript and focus of the study. One thing this saga does well is to illustrate the complex and non-linear nature of... 1/

Moreover I fear that we don't have good evidence that these science reforms — now mandated or strongly encouraged by numerous funding agencies across the sciences — are effective in even in the area of social psychology where there were birthed.

Over the past year, I've resisted blogging about why Protzko needed retracted. Unfortunately, the authors have used their platforms to mislead about the cause for retraction. As they cannot be honest, I'm choosing to be fully open and transparent. #metascience joebakcoleman.com/blog/2024/pr...

Wow! You don't want to overstate it but this does seem big considering the author list and the hype generated when it came out. Just went back to look at the QTs of the original announcement on Twitter: - "wonderful paper", "great and inspiring", "science works if you do it right" etc

Might as well make my TOP hot take: I think it functions as regulatory capture for psychology and related disciplines. The specific recommendations are particularly amenable to the way in which psych conducts science, addresses its more salient concerns, and assume its inferential goals.

Long overdue, but just before my annual leave this paper came out: "Scandal in scientific reform". I ask whether we need scandalization to initiate and sustain reform. The quick answer: sometimes, yes, but less often if we sort out our institutions. A thread. 1/ #sts #metascience #reform

I'll be part of an online panel tomorrow at the University of Sheffield's OpenFest, discussing Open Research, Academic Labour and Equity with @felicitycallard.bsky.social, Mollie Etheridge, and Pen-Yuan Hsing. 15.15 UK time - Register here: docs.google.com/forms/d/e/1F... #Openresearch #metascience

Looking for a new job developing and coordinating a portfolio of metascience-related research? Apply for the role of Senior Analyst in the UK Government’s new Metascience Unit! www.civilservicejobs.service.gov.uk/csr/jobs.cgi... 🧪

Delighted to have received a SIPS commendation award for this project led by my previous PhD student Dr Kathrin Diconne! KAPODI, the comprehensive database of emotional stimuli sets.

KAPODI is a comprehensive searchable database of freely available emotional stimuli sets from a range of modalities (e.g. audio, visual, faces). journals.sagepub.com/doi/10.1177/... @tomhostler.bsky.social

The top google hit for PSA is the Poultry Science Association and the Philosophy of Science Association is in the third spot somehow proving Feynman's point about ornithology and birds and philosophy of science and scientists.

My new #metascience paper open.lnu.se/index.php/me... - 'Research assessment using a narrow definition of “research quality” is an act of gatekeeping'. An invited commentary (one of many) on @gaertneranne.bsky.social et al.'s proposals for reform of research assessment open.lnu.se/index.php/me...

Now online: over three hours of constructive and engaging critiques of metascience and scientific reform practices, featuring Sven Ulpts, myself, @tomhostler.bsky.social, Stephan Guttinger, Sarahanne Field, @nicolecnelson.bsky.social, and @devezer.bsky.social. Grab some popcorn and enjoy!

Critical metascience blog post with a few notes on the recent COS symposium “Critical Perspectives on the Metascience Reform Movement” with presentations by @penders.bsky.social, @tomhostler.bsky.social, Stephan Guttinger, Sarahanne Field, @nicolecnelson.bsky.social, and @devezer.bsky.social 👏

Looking forward to talking at this event next week about how open research practices may increase administrative burden for researchers, and how this work may go unaccounted for in academic workloads. #openresearch #openscience #metascience