I think some of these points miss the mark. Moreso, I'm concerned that they'll be used by others to smuggle hidden moderators debates back into psychology. 🧵 www.bps.org.uk/psychologist...

Here's my conversation with Mu Yang on Metascience Matters: www.youtube.com/watch?v=E2EK... We discussed her work as a scientific sleuth, academic incentives for positive data, individual cases she has pursued, and why she loves being a sleuth. Also on Spotify: open.spotify.com/episode/16R6...

I built an R package that turns Shiny apps into UIs that render directly inside Claude Desktop or ChatGPT. It's called shinymcp. Drop-downs, plots, tables all inline in the chat. github.com/jameshwade/shinymcp

The tidyverse just got even nicer to use!

Basic questions like "how many participants are in the dataset?" can produce surprisingly different answers between analysts. Results from students in my class in data wrangling in tidyverse, who are good at wrangling but still have to make semi-subjective choices:

Are female economists treated differently than males in academic seminars? These authors wanted to know whether gender shapes how scholars are treated when presenting research. So they built a massive dataset of 2,000+ economics seminars, job talks, and conference presentations from 2019–2023...

Don't you f**king dare.

If you're interested in data sleuthing but aren't sure where to start, or if you're conducting a systematic review/meta-analysis and want to ensure you're not including junk studies, check out this Cochrane training session on Trustworthiness Assessment by @jdwilko.bsky.social

My own personal example of this for a study I got rejected in one place as a reviewer and then got published at another, unchanged. Look at the two citations in the image. The years alone should give away they don't support the claim made in the paper. www.sciencedirect.com/science/arti...

Wow this scoring chaos seems to be an extreme case of what I say about many ad hoc analyses: no derivation of method from a clear scientific theory, no assessment of statistical properties, and decades pass before someone notices. This happens in biology too, so let’s not pick on psychology only

But data is available from the authors upon reasonable request. For a meta-research paper this is simply ridiculous Honestly, read our paper more pubmed.ncbi.nlm.nih.gov/35623542/

"One application per year" will have massive consequences. Given the current funding rates, labs will easily be out of SNF funding for several years in a row.

8.0% of citations in the medical literature contain major errors. E.g., the cited work makes the opposite claim or is unrelated to the claim in the citing article.

"you could - by careful choice of an existing scoring method from the literature - find any effect, or nothing, or the reverse of any effect you choose. This is bonkers. ... so extreme as to be farcical." This is the sentiment we were hoping people would come away with! w/@anniria.bsky.social

We spent time checking how outcomes are registered and reported in publication, using database of RCT in rheumatology. The results just got published in Seminars in Arthritis and Rheumatism: www.sciencedirect.com/science/arti... Results aren't good ⬇️1/5 #MetaResearch #ClinicalTrials #Rheumatology

Both kids are gone so naturally, I'm making the most of my precious free time.

Great talk from @lhdjung.bsky.social at the SwissRN Research-On-Research day on his, our, and other folks' work to develop trustworthiness assessment tools. Finally his {scrutiny} R package has a logo!

Fortunately, Conflict of Interest begins at $200,001

And “gaming” is the term that the gambling industry **heavily** pushes because it sounds more innocuous The Responsible Online Gaming Association? Industry initiative to encourage safe gambling International Center for Responsible Gaming? Industry funded scientific organization that funds research

It will still take decades before social sciences understand with what Wittgenstein tried teach them 70 years ago bsky.app/profile/mkar...

What happens when you skip the measurement groundwork and jump right into running "real studies", exhibit 7693782

First post of the year, new paper out today: we present possibly the biggest case of systematic Measurement Schmeasurement in tech use. It seems that most studies on gaming (videogame) addiction/disorder haven't measured gaming after all. This research took years, so long 🧵 doi.org/10.1098/rsos...

The only just so story here is the claim that there’s no evidence for incentives shaping scientific practices. I’ve supplied lots of evidence, nudged for reply, DMd, waited; no response. I look forward to reading the book.

"The research literature is an unsafe workplace" by @jabyrnesci.bsky.social and @aidybarnett.bsky.social Really insightful framing! www.tandfonline.com/doi/full/10....

Foody et al 2013 has over 130 citations (top 1%) as support for an element of Acceptance and Commitment Therapy's model, but it contains no evidence for that claim. This is a case study in flawed original studies being cited favorably and uncritically until it becomes a common scientific belief.🧵

Self-report scales in psychology get a lot of flak & there’s a desire for more objective, “scientific” behavioral measures. Now if you look at some of these more closely, there are hundreds of way to score them to generate some outcome variable — and many scorings are used exactly *once.*>

We looked at 100 IGT studies and found abysmal reporting standards, and 244 different scoring techniques. Applied to a new dataset, the average correlation between these 244 techniques was 0. 5/15 IGT meta-analyses we looked at did not account for this hetereogenity.

"Social media is like a Skinner box" is a phrase I've heard repeated a lot, but never meaningfully engaged with. We try and do so in this preprint. Behaviorist principles are very useful to understanding digital behavior, but work in this area tends not to be aware of them. So, we provide a primer.

I feel very conflicted about this advice: Can you skip parts of papers? Yes! Reading intro+disc first, and methods only optionally, is completely wrong though. IMHO, intro+disc are often lazy world building. Instead, read the methods first to decide whether the rest is even worth looking at.

Keeping this for the next time someone says that it is meta scientists who engage in baseless attacks or have a tone problem, rather than people who clearly haven’t read the work and its demonstrations and justifications of what is problematic to what end.

The Iowa Gambling Task is an extreme example of Jingle Fallacy and schmeasurement. In 100 articles we found 244 different ways of scoring it, 177 were never reused. Correlations between them range -.99 to .99. At the same time, we show meta-analyses combine these results as if they’re equivalent.

Thanks to @ploederl.bsky.social for leading this comment on an RCT with many issues

News from scientific self-correction: Authors pushing to get errors in their papers corrected. Lukas continues to be a role model for how scientists should handle post-publication peer review.

Improving scientific practice can seem daunting. In this fantastic talk (and thread below), Julia Rohrer shares practical ways to communicate methodological insights to a wider audience of researchers.

RegCheck is one of our group's most exciting projects IMO, and I'm particularly happy to see the codebase now be open scourced!

Comparing registrations to published papers is essential to research integrity - and almost no one does it routinely because it's slow, messy, and time-demanding. RegCheck was built to help make this process easier. Today, we launch RegCheck V2. 🧵 regcheck.app

Finally, after some exchange with the editors who requested the raw data (revealing more oddities), the paper was retracted. link.springer.com/article/10.1...

Look at the forestplot 😂 You know, shit and chocolate... www.cambridge.org/core/journal...

wake up babe, new effect size metric just dropped

Now the picture looks quite different! Clear increases in loneliness for younger cohorts, possibly even declines for older cohorts. However, if we go from left to right for an individual cohort, not only survey year increases -- everybody also ages one year per year, by definition.>

Great news! The unethical hepatitis B vaccine trial planned for Guinea-Bissau has been cancelled. Public pressure and insisting that clinical trials pass ethics analysis worked. www.theguardian.com/us-news/2026... 1/2

"The rewards to 'discovering' a spectacular scientific finding [in psychology] are large; the rewards to debunking frauds or deflating exaggerated claims are small if not non-existent. If these are the rules of the game, we should not be surprised at the way the game is played."

Academic life is just repeating this to one another until we retire @jamiecummins.bsky.social

📽️ "Taboo topics we can no longer avoid: negligence, tampering, and fraud" / with Ian Hussey @ianhussey.mmmdata.io youtu.be/sdRRafephyM?... The event was organised by ReproducibiliTea UniBasel @swissrn.bsky.social

Want to get the data out of a PDF figure? As in, the actual data – not a rough trace-along-the-lines version? I made an app you might like: adamkucharski.github.io/pdf2plot/ It all started a few years ago... 🧵

> "There is no suggestion of malpractice" The article's pubpeer thread would strongly disagree. pubpeer.com/publications...

This is good. Authors ask an LLM to write a systematic review. The LLM states ‘prospectively registered’ and includes a fake registration number. When queried, authors send a corrected reg number…which has been created after the query. You love to see it. Happy new year!

Tomorrow I’ll be speaking on a panel for the WHO webinar Social Health and Digital Play: A Conversation, Not a Conclusion. I’ll be sharing insights from a recent Delphi study focusing on priorities and future directions in video game research. If this topic is of interest, feel free to join!