⚙️ New on the COS blog: Principal Research Scientist Noah Haber (@whaleactually.com) shares how the SCORE project developed a collaborative reproducible manuscript workflow, walks through two different approaches, and introduces an open-source R package. www.cos.io/blog/toward-...

For most scientific papers, coathors collaborate in the manuscript doc and someone manually enters analysis results/tables/figures into it. In this case, reproducibility effectively stops at the data and code. What this blog post presupposes is, what if it didn't? www.cos.io/blog/toward-...

When I started working in global health a little over 10 years ago, infectious disease (particularly policy) was said to be a declining field to be almost entirely replaced with chronic disease. That ... did not happen.

Had a small part of this that I am pretty proud of: Full push-button reproducible manuscripts from data to manuscript. Nearly every stat, figure, number, table, etc is pulled directly from analysis code and data and "knit" with manuscript text written in Google Docs. Blog post coming soon ...

"Elapsed time" is to Intention to Treat as "Moving time" is to Per-Protocol. I will not explain any further.

A few of the first recruitment attempts to US-based journals have been met with a "times are too uncertain to take on a project like this," as expected Small potatoes in the scheme of things, but just another example of the astounding destruction of scientific progress happening right now.

The "meta trial" idea is maybe the most ambitious thing I've ever gotten to try in real life. In theory, it goes a LONG way towards solving the feasibility, logistics, and incentives problems inherent in multi-unit policy experiments at scale. Not just for journal policy. For policy period.

This meta trial idea is pretty cool. Eventually you could arrive at a situation where journals help researchers get individual credit for research labor on large collaborative projects

Hi folks! Want to reintroduce a thing I'm leading at @cos.io: the Registered Revisions (meta) Trial. This project about new peer review policy and a WILD new way of doing actionable evidence generation via RCTs. A LOT of RCTs. Now piloted and ready for the main stage, and looking for partners! 🧵👇

Hi folks! Want to reintroduce a thing I'm leading at @cos.io: the Registered Revisions (meta) Trial. This project about new peer review policy and a WILD new way of doing actionable evidence generation via RCTs. A LOT of RCTs. Now piloted and ready for the main stage, and looking for partners! 🧵👇

death to "is it A or is it B that causes Y" long live "how much do A and B contribute to Y"

Triangle folks: I've got water, a bunch of 5 gallon jugs, tools galore, and plenty of time to deliver food/water/whatever to where it needs to go. Throw a DM my way if you need anything at all.

Welp, it finally happened: I messed up a thing in a study involving participants, caught it, and am working on submitted a complete report and mitigation plan of the issue to the IRB. Fortunately the damage to my pride is almost certainly larger than any risk to participants. But still. Humbling.

o hai, i am in copenhagen, sciencing yur sciences

Gonna be hanging out at the International Conference on the Science of Science (ICSSI) in Copenhagen next week, chatting about weird meta experiments, repping @cos.io, and generally trying to be useful in all the chaos. If you're around, come hang out!

RCT-only plus liberal use of guidelines for power, measurement error, heterogeneous treatment effects, examining attrition, etc. We like to set up methods as the gold standard but then put little back doors in rather than own up to our true uncertainty.

Remember folks: the proverbial "gold standard" for evidence was driven by our own mainstream institutional evidence based medicine movement. A simple (read: stupid) set of rules dictating what's good (double blind RCT, meta-analysis, etc) and not (anything else). What could possibly go wrong?

Added my name. There is a lot to be said about how the aesthetics of science and scientific critique are beiung twisted, coopted, corrupted, and targeted for destructive ends. Right now, it is time to make yourself heard, however you choose to do it. actionnetwork.org/petitions/op...

Coming soon: the most fully realized preregistration/registered report ever (I think) * Pre-written methods and results section * Full reproducible data -> manuscript text and fig pipeline * Pre-populated data w/ full trial(s) simulation * Multiple levels (trials + meta analysis) for all

how it started / how it's going www.science.org/doi/10.1126/... www.aaup.org/news/cowardi...

We are experiencing an assault on science unparalleled by anything I’ve seen in my life. It’s not one issue or another anymore, the entire institution is under attack by the most powerful individuals in the country. This Friday, where will you be? standupforscience2025.org

These two takes are VERY different. Both identify the severity of the threat we face. But when it comes to what we should to do about it, one is a call to fight alongside the students and postdocs; the other takes the opportunity to tell those students to stop being critical of university admins.

Good news: Added a new use on a slide for how preregistration might enhance research, yay! Bad news: It's a more robust public record of your research if, say, some capricious and vindictive government decides to pull your funding, force modifications, censors, or cancels it altogether.

The practice of science is rejecting and challenging authority, not being obedient to it. Science is punk.

The problem is that some data always lie while some always tell the truth

Absolutely not. This time calls for an array of approaches, righteous (and just plain right) fury included. This take is somehow simultaneously entirely predictable and surprising. It's profoundly demoralizing and ineffective, maybe harmful. www.science.org/doi/10.1126/...

This is fair, but remember if this is something new to you, or just becoming apparent, there's a wide body of research, activism, resistance, stories and archives to draw on to make your case when challenging weak institutions and encouraging colleagues and friends to join you.

If we have any hope of preventing the collapse of US scientific and research environment, we also have to rip off our blinders on the ways our institutions are weak and how they have failed so many. Those weaknesses are points of leverage for people trying to tear the whole thing down.

When you are imagining a plucky smart investigator epidemiologist going to who knows where to investigate an outbreak like in the movies, you are thinking about the Epidemic Intelligence Service (EIS). I would strongly recommend avoiding the potato salad for a long while.

i just discovered that you can change your handle to a web domain you own, and, um

Yes and therefore messaging about the overall economic impact mostly just reproduces existing partisan cleavages. But many rural areas are connected to a blue or purple metropole through (specialty) healthcare and to the extent that cuts reduce quality or increase cost maybe that moves the needle. 🤷‍♂️

To understand this moment for the NIH cuts, you need to need to understand which CITIES will lose, not which states or fancy universities. The cities that are largely blue bubble, more affluent, cosmopolitan, liberal, higher education, exclusive, diverse, etc i.e. exactly the intended targets.

The best I can offer as advice to the incoming generation of scientists. The most important one BY FAR is finding your people. The subtext for that is to also understand who will not be your people.

I am giving a seminar on research integrity and ethics in the context of institutional research systems to med students on Monday. What am I supposed to say.

I've already seen a couple of "how could they make a stupid mistake like this, don't they know this will destroy research and higher ed?" It's not a mistake or an unintended consequence of a different goal. The aim is to destroy research and higher ed.

Auto-rejecting all grants that contain banned words would be bad, but it is not happening. Flagging grants for additional review based on discriminatory criteria IS bad and IS happening! We can't make as much progress fighting a real bad thing if some people are fighting a fake bad thing instead.

my coping mechanism is acquiring bikes

This is the same scheme as PNAS, where club members get a "contributed track" where they nominate their own reviewers and auto publish more or less anything. It's horrible at PNAS, and it's horrible here. Failing to fix the cracks in our already broken system made it so much easier to destroy.

You as non-editor researchers can also have influence here! Write to your journals' editorial board or EIC, asking them to make explicit a policy that they will not comply with these requests. Sometimes all it takes is a quick email for an editor who didn't think about it or to prioritize it more.

"We will not retract published articles on request by an author on the basis that they contain so-called banned words" @bmj.com takes a stand: "Medical journal editors must resist CDC order and anti-gender ideology" www.bmj.com/content/388/...

If you're not in the DC area, you should be organizing where you are. But if you are in the DC area, I think you're going to want to come to the Treasury building today at 5pm. It will be newsworthy. www.mobilize.us/moveon/event...

Reuters is now covering the order for all CDC scientists to withdraw, take their name off of, or edit all existing publications in consideration if it has such evil words as "prenant people" This is research that is done and written. Straightforwardly censorship. www.reuters.com/world/us/us-...

seeing takes about the aid institutions and their role in imperialism. correct. If someone said "investor-owned hospitals exploit the vulnerable to benefit shareholders" I'd nod. If their followup was "and that's why I'm turning off your mom's dialysis machine" I'd...have some followup questions

I’m watching closely for anything happening inside BLS, Census and the other federal statistical agencies. Know something? Reach out — anonymity guaranteed. Signal: bencasselman.96

Science is political. It wouldn't necessarily have to be partisan. But when one party is opposed to science-based policy, scientists have a choice between saying "you're right, thoughts and prayers are probably as effective as reducing carbon emissions" or saying "fuck that."

I have a side hustle making a unique woodworking tool. It is designed, assembled, calibrated, shipped, serviced, etc by me personally in Durham, NC, USA. I am pausing it, simply because this very American business can't operate with under trade war tariffs. Here is how those tariffs actually work:

One might call this preregistering your position for an impending potential fight.

Journal editors: time to step up to the plate.

Endorsed, both as a scientist concerned with science integrity and as current research scientist at the Center for Open Science (albeit on the research side, so I don't do much / any work on OSF.io itself). You can bet that the team behind OSF is on it.

Hey if anyone is considering backing up any datasets for some reason today, the Open Science Framework has free storage up to 5G/private repository, and up to 50G/public repository. (The OSF is run by a 501c3 (Center for Open Science), so if you do this you might also consider making a donation.)