Comment by the STRATOS initiative: (including @boulesteixlaure.bsky.social @mhstat.bsky.social) onlinelibrary.wiley.com/doi/10.1002/... (6/8)

Workshop on “Evidence & Uncertainty in Science: Methodological, Philosophical and Meta-Scientific Issues” 10th & 11th June, Uni of Tübingen, in-person only uni-tuebingen.de/de/281247#c2... ‪With @cruwelli.bsky.social, @boulesteixlaure.bsky.social, @babeheim.bsky.social, & @hendriks.bsky.social ‬

NEW (METASCIENTIFIC) PREPRINT on the exploratory/confirmatory distinction: Title: On "confirmatory" methodological research in statistics and related fields by @iamjulianlange.bsky.social J. Wilcke, @sabinehoffmann11.bsky.social M. Herrmann and myself arxiv.org/abs/2503.08124

NEW METASCIENTIFIC PREPRINT: "The impact of the storytelling fallacy on real data examples in methodological research" by M. Mandl et al: arxiv.org/html/2503.03... or why it is misleading to argue in favor of a method just because one can tell a nice story on the results obtained for n=1 dataset.

NEW PAPER: Confidence intervals for (e.g., cross-validation) prediction error by H. Schulz-Kümpel, S. Fischer et al. "Constructing confidence Intervals for “the” Generalization Error – a Comprehensive Benchmark Study" openreview.net/pdf?id=x7kCj...

This Registered Report masterpiece just dropped at BMC Biology, brilliantly led by a great team with the help of 300+ analysts & reviewers Same question, same data: go figure! tl;dr: Substantial heterogeneity among results comes from differences among analytical choices 🔗 doi.org/10.1186/s129...

IMO it’s a mistake to give stats to 1st year med students. It’s not why they chose medicine & they resent it. Better to wait until they have developed some curiosity for it. I argued unsuccessfully for this during my time at UCL.

Does #randomization ensures balance of risk factors between groups? Consider this: In Denmark 860 individuals were randomly allocated to either intervention or control. Individuals were unaware of their allocation. No intervention took place. Mortality was higher in the intervention group (p=0.003)

So happy to see you all here! As the @lmu-osc.bsky.social coordinator, I just started a new project: coaching individual research groups so members can switch together to #OpenResearch - a tailored pedagogical intervention to maximise chances of sustainable adoption in the group, and a lot of fun! 🧵

Something that's been bugging me for a while in bioinformatics data analysis is this overreliance on packages, workflows and what's been called "cargo cult science". Can we have more conceptual thinking, more theory? Asking for what we really want to achieve and what we need to do gets us there.

Happy to have been involved in this exciting project on the registration, design and reporting of statistical simulation studies, with @bsiepe.bsky.social @timpmorris.bsky.social et al., appeared in Psychological Methods:

AI researchers: hold my beer — Old model performed slightly not worse when data were generated by the new generative AI method I just made up

I made one for stats papers

Simulation studies are essential for methods research. How well are they conducted & reported? How can we improve their quality? Out now in Psychological Methods, see 🧵 below. With @fbartos.bsky.social, @timpmorris.bsky.social, @boulesteixlaure.bsky.social, @danielheck.bsky.social & Samuel Pawel

MEMTAB 2025 pre-conference courses news! 2 options: - An Introduction to Clinical Prediction Models and Sample Size Calculations for Model Development & Evaluation - Systematic Reviews of Prognosis Studies Just £50 when registering for conference! Details 👇 uobevents.eventsair.com/memtab-2025/...

We must stand against the arbitrary categorization of continuous variables! ... and that's why I'm proud to announce my support of abolishing time zones in favor of the time gradient

Which reminds me of this Andrew Vickers quote www.nature.com/articles/ncp...

Independent GroupLeader / PI position in AI in Genome Biology, Multimodal Omics at European Molecular Biology Lab (EMBL), Heidelberg! www.embl.org/jobs/positio... Looking in particular for researchers with quantitative/ methodological background who want to dive into leading edge biology research.

For all the new followers here: welcome! Here some recent highlights: (1/4) My eclectic and subjective list of scientific writing tips

Bias in the evaluation of female academics is hard to remove www.forbes.com/sites/kimels...

Created a new group replacing and updating a list I enjoyed following on the bird site. Mostly people posting on medical stats and DS/AI ICYI: go.bsky.app/ArqEz36

And there, look out! A meteor! In the form of the BMJ that code sharing will become mandatory. We told you this was coming. Scrutiny of your research data practices. And now it's here. www.bmj.com/content/384/...

Heinze G, Boulesteix A-L, Kammer M, Morris TP, White IR. ‘Phases of methodological research in biostatistics—Building the evidence base for new methods.’ Proposes a ‘methods-development pipeline’… 10/ doi.org/10.1002/bimj...

Friedrich S, Friede T. ‘On the role of benchmarking data sets and simulations in method comparison studies.’ Nice discussion of simulation studies vs. benchmarking datasets for prediction/classification methodology work. 9/ doi.org/10.1002/bimj...

Strobl C, Leisch F. ‘Against the “one method fits all data sets” philosophy for comparison studies in methodological research.’ Makes the argument for simulation/comparison studies to ask ‘which methods work well when’ instead of ‘which method is best on average’. 8/ doi.org/10.1002/bimj...

Morris TP, White IR, Cro S, Bartlett JW, Carpenter JR, Pham TM. ‘Comment on Oberman & Vink: Should we fix or simulate the complete data in simulation studies evaluating missing data methods?’ Says when we can fix the complete data and gives many cautions against doing so. 7/ doi.org/10.1002/bimj...

Oberman HI, Vink G. ‘Toward a standardized evaluation of imputation methodology’ Loads of useful and practical advice for simulation studies comparing missing data methods (particular relevance to missing covariate values in observational research). 6/ @oberman.bsky.social doi.org/10.1002/bimj...

Pawel S, Kook L, Reeve K. Pitfalls and potentials in simulation studies: Questionable research practices in comparative simulation studies allow for spurious claims of superiority of any method. Nice table of QRPs, and recommendations to avoid them. 5/ doi.org/10.1002/bimj...

Nießl C, Hoffmann S, Ullmann T, Boulesteix AL. ’Explaining the optimistic performance evaluation of newly proposed methods: A cross-design validation experiment’ Shows how method comparison studies can over-claim and gives some meta-evidence… @boulesteixlaure.bsky.social 3/ doi.org/10.1002/bimj...

This is hands down my favourite issue (now known as 'collection') of any journal ever. I love the 12 meta-scientific papers. So happy that @boulesteixlaure.bsky.social masterminded it. A thread of some personal highlights 👇 1/ doi.org/10.1002/bimj.202400031

Glad to see so many people interested in stats here :) I would be happy to discuss "neutral comparison studies in methodological research", see the editorial of our special collection in Biometrical Journal: onlinelibrary.wiley.com/doi/full/10....

1) people don't want to read large docs, so get AI to summarize them 2) people don't want to write large docs, so get AI to expand their summaries 3) We're very close to having AIs writing documents just for each other to read 4) AI #1 to AI #2: "WTF we doing here, man?"

People often rush to submit an article to a special issue related to their work? Why? What difference does it make if your article is in a themed issue? Do people think it increased chance of acceptance?

For the one I was involved with, it’s now an easy-to-remember place containing a lot of good & relevant papers. I’ll remember the issue where I might not remember every individual paper. onlinelibrary.wiley.com/doi/toc/10.1...

"Why Student Ratings of Faculty Are Unethical" Food for thought (curious what others think). #AcademicSky link.springer.com/article/10.1...

PREPRINT: "On the handling of method failure in comparison studies", by M. Wünsch with @timpmorris.bsky.social Imagine we want to compare the performances of methods A and B, and method A fails to produce results for some of the simulation iterations/data sets. What next? arxiv.org/abs/2408.11594

NEW PAPER published in BMC Med Res Meth, by Maximilian Mandl, with S. Hoffmann et al. on the connection between multiple testing adjustment and p-hacking/researchers' degrees of freedom: bmcmedresmethodol.biomedcentral.com/articles/10....

New role for a Project Coordinator at COS. Perfect for an early career researcher passionate about open scholarship and looking to gain research experience, whether to then go to grad school or stay in industry. jobs.lever.co/cos/ce551e5d...

New preprint! "Prevalence of transparent research practices in psychology: A cross-sectional study of empirical articles published in 2022" osf.io/preprints/ps... After > decade of new infrastructure, advocacy, & policy, how often are transparent research practices used in empirical psychology?

We've got a new DP: J. Fitzgerald implements equivalence testing methods in a systematic reproduction of 135 null claims made in 81 articles from top economics journals. econpapers.repec.org/paper/zbwi4r...

No. This is not acceptable. The methods section is the single most important part of a scientific paper. If those details are relegated to supplementary materials, then it's not a scientific paper in a scientific journal.

"A random half of panelists were shown a CV and only a one-paragraph summary of the proposed research, while the other half were shown a CV and a full proposal. We find that withholding proposal texts from panelists did not detectibly impact their proposal rankings" link.springer.com/article/10.1...

Join our workshop: An introduction to using simulations for study design and pre-registration, by Joel Pick When: 30th May, 10:00 am CEST This workshop is aimed at empiricists from any career stage or discipline. Register at events.humanitix.com/sortee-works... #SORTEEtools

Happening today at 16:00 (CET)!

Exciting news: I will be teaching a 4-day summer school on open science this August! The course is for ECRs in the social & behavioural sciences who want to get up to speed with open & reproducible practices and/or learn how to handle specific problems. utrechtsummerschool.nl/courses/data...

CSAMA one-week summer school biological data science, in Brixen/Bressanone, South Tyrol, 23-28 June. Register now for one of the last places. Topics include single cell omics, statistical foundations, R/Bioconductor tools, mass spec, interactive workflows csama2024.bioconductor.eu

We are planning randomized controlled studies to assess a causal effect of #statcheck, specifically. Are you an editor & is your journal interested in collaborating? Let me know! 7/n

Happy to be part of the instructor team at CSAMA in Brixen (Italy), 23 - 28 June 2024. You can still register! csama2024.bioconductor.eu

Bear with me for tooting my own horn but that's why we wrote this paper and that's why we need a principled, theoretically literate approach to metascience. Fewer heuristics, deeper understanding, measured claims. royalsocietypublishing.org/doi/10.1098/...