Apropos of something I'm working on right now: It's great to use Claude to break down technical walls and perform tasks that you wouldn't have otherwise been capable of. But you MUST still fully understand what you're trying to do and how you (and Claude) are doing it. You need to be driving.

Just 17 more people to make history/a nice round number

If you don't know how reliable your measure is, you're wasting your participants' time. Ch 8 of Experimentology argues that measurement reliability and validity deserve more attention than most experimentalists give them. 🧵 experimentology.io

a couple of hours before my keynote, I went through an intense negotiation with the organisers (for over a hour) where we went through my slides and had to remove anything that mentions 'Palestine' 'Israel' and replace 'genocide' with 'war crimes' 1/

A recurring conversation: - I asked AI [an LLM] to do a complicated thing that I couldn't have done myself and don't understand, and it did it - how cool is that?! - How did you work out whether it got the right answer? - 🤷‍♂️ - 😢

The Glasgow Tower yesterday evening as the sun was going down behind it. #glasgow #glasgowsunset #glasgowtower #architecture #architecturephotography

Our metacheck package now has an online manual! In 29 short chapters we explain how to set up metacheck (running all modules 100% locally if you want), and explain the 18 available modules. www.scienceverse.org/metacheck_bo... We are still in beta, and would love to get feedback!

Periodic reminder that publishing papers is not the goal in and of itself. The paper is a mechanism for sharing what you have learned--sharing processes and outcomes that others should be aware of/should think about too. 1/3

The machine just fucking butchers informed consent and leaks PII in a way that would instantly get *you* fired, but it ends up getting your research budget instead.

I am home sick for the first time in ages and watching YouTube and saw the most cursed add for “Peterson University”. Yes, that Peterson :( WTF 😬

People always say "tell a story" but then don't explain how to do so! How is that helping? If we knew how to tell a story we wouldn't need the advice in the first place. 1/

Tip for senior authors, too…

Tip for junior authors: A power analysis is only as good as the effect size you feed it. "Medium" is not a justification. Tell us whether your effect size comes from prior empirical work (justifying that choice) or from theory (e.g., the smallest effect you'd care to detect). #cogpsyc #AcademicSky🧪

Behind almost every research study is a collections specialist, data manager, technician or bit of code that never makes the paper. The Hidden REF 2026 is open & recognises exactly those outputs & roles, the ones traditional assessment leaves out. hidden-ref.org/hidden-ref-c...

Is it any less weird if it's a hollow face illusion?

reading AI eval papers using llm-as-a-judge is alarming for many reasons; many “scientific claims” rely on evals executed on models that are now deprecated. how are we supposed to treat “findings” that are inherently not reproducible? i despair. llms are accelerating the decay of scientific rigor

Sports Commentary xkcd.com/3262/

This is worse - you need a Bayes factor to interpret a null effect

I was very happy to see that Royal Society Open Science just used my @prereview.bsky.social like a traditional review. When I was asked to review the manuscript, I instead did a PREreview (public preprint review) and put nothing but the link to it in my review. prereview.org/reviews/2049...

This is a conclusion that anyone working on automated assessment of faces could have predicted without seeing the tech at all!

A clear example of how LLMs inherently systematically erase those who are already systematically disadvantaged.

Am I overreacting by being incredibly annoyed at being asked to create a new account (and fill in a bunch of personal info) at a journal or foreign grant funder just to tell them that I will not review something with a 3-week deadline and/or completely outside my area of expertise?

Is anyone else losing the will to live reviewing metascience grant proposals this week? Please, I am begging, if you must use genAI to help write a grant proposal, at least ruthlessly edit out the sentences full of jargon and empty of content!

It's that time of year when the JIFs are out. Over-reliance on it fuels integrity risks like papermills & fraud. Everything is connected! Join us in shifting the focus from venue prestige to merit. Explore RRA resources & course: sfdora.org/reports/ & bit.ly/DORA-Intro-Course

Something that may be difficult for non-experts to grasp is that experts can recognise when a student’s text is not meaningfully connected to cited literature in the expert’s area of expertise.

from now on, philsci archive only accepts submissions from institutionally/socially recognized philosophers. this is one of my predictions of how llms change the public space, free-floating text becomes less useful, securely signed documents more important philsci-archive.pitt.edu/policy.html

In 1908 the Lancet called for an 18+ age limit on reading in bed for U.K. children.

Looking for feedback on my data skills assessment plan. Has anyone tried face-to-face code review as a method of assessing understanding? What pitfalls might there be? (I usually have about 60 students and do all the marking for the final)

Is the Microsoft Office Ribbon causing people to have fewer children? A provocative new working paper explores the persistent drop in birth rates since Office 2007 was introduced nearly two decades ago.

Aren’t 16-year-olds allowed to join the army and go to university?

Took some time to slow down and write some good documentation and tests for a little side project, and as usual this process highlighted several areas for improvement. I can't think of a time this *hasn't* happened.

I built a website that tracks open source tools that provide automated checks for scientific manuscripts: lakens.github.io/automated_re... It provides a summary through RSS feed that you can subscribe to. It tracks 13 repositories - if you want your repo to be included, let me know!

So, naive question about these campuses that pay OpenAI $13mil/yr. Why don’t they pour that money into a cluster running the strongest available open-weight model, with API and interactive options. And if the answer is “overhead,” why aren’t we collaborating?

🏳️‍⚧️🏳️‍🌈 such a pretty campus @glasgow.ac.uk

It’s time to start thinking about how to change my MSc data skills assessments for next year. Last year they did 4 output reproductions (write code to make an output given raw data) of increasing complexity. The marking was too much and AI use prevents learning. psyteachr.github.io/reprores-v5/

My much more radical opinion is that all ethical approvals for human/animal biomedical research should be openly published, and all papers resulting from those studies should be easily linked to the approval. This one *really* freaks people out and I suspect we all know why.

This is my nightmare scenario for metacheck. We’re building it to support human decision making, never replace it. This is clear in our values, but how can we ensure users follow these values? www.scienceverse.org/metacheck/#o...

From 500 views in the first two months to over 100,000 views in the last few days. The ADA opinion piece that was too scary to be distributed at the ADA meeting has gone full Streisand. Y'all are doing the Lord's work.

What happens when researchers stop competing for funding? Twenty Dutch metascientists replaced competition with collaboration for NWO grant applications, proving collective approaches can outperform individual ones. @sajedehra.bsky.social @lakens.bsky.social www.tue.nl/en/news-and-...

It’s good — essential even — to marginalize wrong and bad ideas. The way we get out of this is by pointing out how stupid and evil these inbred troglodytes are, not by feeding their ass-backwards superiority complex.

My PhD Sajedeh Rasti has published a World View piece in Nature today: "Don’t compete, collaborate: why collective funding applications are the future" www.nature.com/articles/d41... She discusses how she successfully brought together a team of meta-scientists to collaboratively apply for funding.

The @glasgow.ac.uk 2026-9 Research Culture Action Plan has launched! The core values here are admirable; now we keep working towards truly living them. 🔍 Research Integrity & Ethics 📖 Open Research 🏆 Research Recognition 👩🏿‍🤝‍👨🏾 Collegiality and Teams 🎓 Career Development www.gla.ac.uk/myglasgow/ri...

Just found this intriguing new function by Wadea Abu Dahoud for making a log-scale axis squiggly, mostly to act as a visual cue/speed bump that the axis isn't linear. github.com/wade31985-ar... #rstats #dataviz

We're excited to launch J·ROR, the Journal of Research on Research. A new open-access home for research on how research is funded, organised, conducted, communicated, and evaluated. Our first editorial: www.tandfonline.com/doi/full/10.... #ResearchOnResearch #MetaScience #OpenScience #STS

0:41 for my cameo and a closeup of me stacking books like @statmodeling.bsky.social & Hill’s HLM book, The 7 Deadly Sins of Psychology, @kevinguyan.bsky.social’s Queer Data “…and upholding the highest standards of research practice.” (I was instructed to say it very slow :) youtu.be/KSVZSMOUysc

A Mastodon post I saw shared on Discord:

#RStats challenge and potential community project: * Create a minimal package 'dumpsterfire' that triggers as many 'R CMD check' issues as possible! Fun, but could also become a excellent resource for illustrating and understanding package check issues Here's a start: github.com/HenrikBengts...

Nature reports on how labs are teaming up to run a host of huge projects that aim to bring rigour to cognitive science. #cogpsyc 🧪

I’ve always taught students the sandwich method of structuring their writing, with a median of 3 main points in each section or subsection. And I’ve always adored bullet points. 🤖

Remember the current NIH director going on about protecting researchers from government censorship? Today his people called the police in to forcibly remove my colleagues from their own society meeting for sharing an editorial published in their flagship journal that was critical of him. Gift link