so maybe @mdpiopenaccess.bsky.social could make a resolution to remove all editors and reviewers who have accepted nonsensical papers. It would be quite a cull

Journal article: “the data are available on request” The data:

I regularly cite Prinz et al.(www.nature.com/articles/nrd...) as a ref for low replicability in (non-psych) preclinical research. Believe it or not: They've omitted which studies they attempted to replicate! I'm guessing this isn't news to everyone, but it was to me. Bizarre.

The package formerly known as papercheck has changed its name to metacheck! We're checking more than just papers, with functions to assess OSF projects, github repos, and AsPredicted pre-registrations, with more being developed all the time. scienceverse.github.io/metacheck/

Finally, @bjoernhommel.bsky.social's and my paper introducing the SurveyBot3000 is officially out in AMPPS. It's a fine-tuned language model that guesstimates correlations between survey items from text alone. Not perfectly, but useful for search, for example. journals.sagepub.com/doi/10.1177/...

Contrasting to silicon samples, this type of method based on text embeddings seems to be both more principled and more robust in making predictions about scale and item correlations. Super happy to see this great work out!

Great thread about a very timely preprint. Couldn’t agree more with this point especially. When results from LLMs in “silicon samples” can vary substantially as a function of many analytic decisions, systematically making decisions erases much perceived efficiency. arxiv.org/abs/2509.133...

Many think LLM-simulated participants can transform behavioral science. But there's been a lack of accessible discussion of what it means to validate LLMs for behavioral scientists. Under what conditions can we trust LLMs to learn about human parameters? Our paper maps the validation landscape. 1/

At ERROR, we cannot compete with million-dollar bounties for whistleblowers. But it is great to see sleuthing work rewarded, and institutions admitting when their researchers engaged in misconduct.

While we are unable to offer publication in Nature Aging, we believe your manuscript may be suitable for our Gold Open Access sister journal.

Post-publication peer review is at it best when it's thoughtful, scrupulous, steeped in detail – and challenges key claims of the paper. @janhove.bsky.social's discussion of a recent paper on multilingualism exemplifies this.

Did you know that from tomorrow, Qualtrics is offering synthetic panels (AI-generated participants)? Follow me down a rabbit hole I'm calling "doing science is tough and I'm so busy, can't we just make up participants?"

See @jamiecummins.bsky.social 's work

Thanks for sharing! You may be interested in this, which resonates, from @jamiecummins.bsky.social, on the fragility of silicon samples doi.org/10.48550/arX...

A new PNAS paper finds that polarization increased immediately after the release of Lady Gaga’s “Just Dance” and the advent of the late-2000s electro-pop era, which both appeared around the same year, 2008.

Critique argues post-publication peer review "often focuses on minor details" and "risks damaging trust in both the research itself and the peer review process" found to contain: - Hallucinated references - Undisclosed COIs (EiC is author) - 8/9 authors have retractions or related scandals

Congratulations to @bethclarke.bsky.social on her PhD graduation!! It’s been amazing watching you become the incredible scholar that you are, thanks for letting me come along for the ride! You’re the best! 🎉🥂🌟

Keeping this at hand in case I need to point to it and tap

Many psychologists are treating LLMs as if they are the mind of god. This study had chatGPT rate how central academic disciplines are to various constructs. Why would chatGPT know this? Where is the evidence its ratings are reliable or valid? compass.onlinelibrary.wiley.com/doi/full/10....

As someone extremely non-aesthetically oriented, I needed this

Big new blogpost! My guide to data visualization, which includes a very long table of contents, tons of charts, and more. --> Why data visualization matters and how to make charts more effective, clear, transparent, and sometimes, beautiful. www.scientificdiscovery.dev/p/salonis-gu...

Tomorrow: sold-out Bennett Symposium at Jesus College www.bennett.ox.ac.uk/events/2025-.... Day 1 = presentations and panels. Day 2 = collaborative sessions and drop-ins with OpenPrescribing & OpenSAFELY teams. If you're attending, see you there 📊

Project Implicit gives you feedback when you complete an IAT, and the IAT literature has aspired to individual level measurement since its inception. In this article, we quantify the individual level uncertainty around these estimates and show that IAT effects can say very little about individuals.

More research like this on other psychometric scales is needed (although the IAT should just be abandoned regardless). Researchers and practitioners should know the precision of their tools. If someone is 1pt over a cutoff how reliable is this for classification and treatment?

Just published in Behavior Research Methods: The individual-level precision of implicit measures w/ @ianhussey.mmmdata.io 🧵👇 link.springer.com/article/10.3...

So...my undergrad thesis student is doing a quality analysis of studies found in meta-analyses. She identified a few and we contacted the authors to request their effect sizes and other variables for the studies in their papers. Here's what happened: scientiapsychiatrica.com/index.php/Sc...

PCI Statistics and Machine Learning (@pcistatml.bsky.social) is now open for submissions!! statml.peercommunityin.org For an updated list of all the thematic PCIs, visit peercommunityin.org/current-pcis/

Metascientists step up as role models for a healthy error culture in science. Here is a great case where an author and a critical reader collaborated to set the record straight.

This afternoon, I will give a public lecture about #ResearchIntegrity and #ImageForensics, at the University of Bern, CH, where I will receive an honorary doctorate from the Faculty of Science tomorrow. Thank you for your support ❤️

Do LLMs understand humour? My short answer: Who cares. For my long answer, read the article I wrote for our university newspaper: www.uniaktuell.unibe.ch/2025/do_chat...

Ruben is an extremely smart person and also a super nice guy. Work with him!

Want to make nice graphs with me, starting next year? I'm hiring for a position at the University of Witten/Herdecke. uni-wh.softgarden.io/job/61280592...

Fadeout of cognitive training remains one of the more replicable findings in psychology in this preregistered study of 300 preschool children. Well-done study with a 4 year follow up. The language gains either faded or the control group caught up. journals.sagepub.com/doi/full/10....

Very proud to be part of a journal that is trying to be part of the solution 💪🏼 journals.sagepub.com/doi/10.1177/...

1. Transparency is necessary for credibility 2. Transparency is hard to change 3. Require transparency* 4. Transparency is not magic 5. Journals are part of problem 6. Expect more from journals 7. Peer review is not magic 8. A crisis can look a lot like „normal“ science 9. Meta-analysis is not magic

So while it is good that @bmj.com put an Expression of Concern on Attar et al, I wonder what they are waiting for. The deposited dataset matches the reported means, but not the age criteria, & is riddled with duplicated blocks. This should be sufficient for retraction. pubpeer.com/publications...

Voluntary retraction remains a key way to put scientific self-correction into practice. Zhu and Holmes (2024) did the right thing when they realized that some of their results were based on a coding error. Original: psycnet.apa.org/fulltext/202... With retraction: psycnet.apa.org/fulltext/202...

Works the other way around too. “If I was Jonathan Haidt, how would I destroy the next generation?”

Silly for many obvious reasons, but the funniest part of this is that Haidt's books, as well as the vast swathes of discussion about them online, are literally in the training data of these models.

The man who inspired Australia's teen social media ban literally did this:

Many error remain to be found in clinical trials. Patients deserve reliable results. Kudos to these authors for their persistent work to correct the record.

A thread about being wrong: 5 years ago, we wrote a paper about how how newly enfranchised 16-year-olds vote in Austria. But we were wrong. This year, @elisabethgraf.bsky.social, @schnizzl.bsky.social, Sylvia Kritzinger and I are setting the record straight: authors.elsevier.com/c/1juT5xRaZk...

"In 2019 we notified journals about serious integrity concerns in 172 clinical trials. Over five years later, only 22 have been retracted. The 135 unretracted trials have 1989 citations in systematic reviews, clinical guidelines, and consensus statements" [paraphrased] www.bmj.com/content/390/...

If you'd like to learn more about how OpenSAFELY works - and how we solved the privacy and efficiency challenges, to make national GP data securely accessible - here's a 5 minute video! www.youtube.com/watch?v=GRjR...

🚨 SynthNet is out 🚨 Researchers propose new constructs and measures faster than anyone can track. We (@anniria.bsky.social @ruben.the100.ci) built a search engine to check what already exists and help identify redundancies; indexing 74,000 scales from ~31,500 instruments in APA PsycTests. 🧵1/3

I was thrilled to have been invited by @sakshighai.bsky.social to speak to folk at LSE on Wednesday about methodological and inferential issues that have cropped up in social science attempts to study large language models!

Congratulations to @simine.com for winning the Einstein Foundation Individual Award! 🎉 A well-deserved recognition for her seminal efforts to improve scientific rigor, which includes instituting detailed checks for errors and computational reproducibility at Psychological Science.

🏆 Individual: @simine.com, psychologist at @unimelb.bsky.social & editor-in-chief of Psychological Science, is recognized for pioneering methodological rigor, reproducibility & collaborative research, driving initiatives such as @improvingpsych.org & the journal Collabra @ucpress.bsky.social. (2/5)

Next: Jack Wilkinson @jdwilko.bsky.social with 'Problematic clinical trials and the threat to evidence synthesis' Systematic reviews are considered the cornerstone of medicine. But some of the eligible trials that could be included might be problematic. They could get included. #IRICSydney

I think this is an overly pessimistic take from the @bmj.com. Sharing data does not inherently increase trust, rather it enables verification which allows for trust calibration. This example is a win. Serious issues were rapidly detected that would not have been without mandatory data sharing.