When you collect data online, are the results from humans or AI? In a project led by Booth PhD student Grace Zhang, we estimate the prevalence of AI agents on commonly used survey platforms: osf.io/preprints/ps... 🧵

Bots have made their way to Prolific experiments. Our lab has stopped online testing of adults entirely now for this reason - we want to know if what we study is real. Probably data collected 2-3 years ago are ok, but moving forward we just can't know. www.pnas.org/doi/10.1073/...

"Machine learning research is not serious research & therefore hallucinated references are not necessarily a big deal, agrees prestigious group of ML researchers" I ❤️ when the titles write themselves But seriously, I don't think there's much to debate statmodeling.stat.columbia.edu/2026/01/26/m...

Oh Canada!

The results of the RCT didn't support our hypothesis ➡️ The RCT "failed".🤦‍♂️ This is not how science works! #ResearchIntegrity 👍Kudos to Charlotte Strøm for the review www.linkedin.com/pulse/review...

Experimentology is out today!!! A group of us wrote a free online textbook for experimental methods, available at experimentology.io - the idea was to integrate open science into all aspects of the experimental workflow from planning to design, analysis, and writing.

The poll numbers that matter: www.vancouverisawesome.com/local-news/p... @carlbergstrom.com

Seems the White House is using gen AI to write their scientific reports, leading to the hallucinations and mistakes you'd expect. Laughing/crying right now. www.nytimes.com/2025/05/29/w...

Exclusive: National Science Foundation staff were told today that the agency’s 37 divisions—across all eight directorates—are being abolished and the number of programs within those divisions will be drastically reduced.

Climate Terminology Does Not Matter Across tens of thousands of participants in two large-scale experiments, we found that labeling climate change in different ways had no effect on their stated willingness to act. jayvanbavellab.substack.com/p/climate-te... via @dgoldwert.bsky.social

Chegg, notorious resource for student cheating, is mad that Google is copying its answers.

Are you running AB testing studies on Facebook or Google? Or reviewing papers using them? Check out our open access paper, On the Persistent Mischaracterization of Google and Facebook A/B Tests: How to Conduct and Report Online Platform Studies at doi.org/10.1016/j.ij... @boegershausen.bsky.social

IMPORTANT: If you live in the United States or Canada, you’ve probably heard Trump‘s constant claim that the United States is “subsidizing” Canada. It’s wrong of course, but here’s the best explanation I’ve seen about how & why it’s wrong. PLEASE WATCH & SHARE IT A LOT. Kudos to Andrew Chang & CBC.

Underreported: Elon just picked the head of NASA—a massive conflict-of-interest that will damage a government entity that he partners with, depends on, and in some cases competes with. The nominated NASA director is a high-school dropout billionaire who is pals with Musk. He has no business at NASA.

JDM pub: Intertemporal choice questions are usually like "Receive $10 now, or $15 next month?" Yet real-life decisions usually involve a mix of negatives and positives. Do mixed-valence questions better predict real-life behaviours? No! go.shr.lc/3OG36rz @yeli.bsky.social @sjdm-tweets.bsky.social

Black Friday Deal Alert! Enjoy our current atmospheric CO₂ because we'll never see it this low again!

Introducing RegCheck: a tool which uses Large Language Models to automatically compare preregistered protocols with their corresponding published papers and highlights deviations. @malte.the100.ci @ianhussey.bsky.social @ruben.the100.ci @bjoernhommel.bsky.social regcheck.app

the fallout from the Francesca Gino research misconduct scandal isn't over. @engber.bsky.social writes about how an effort by some of her colleagues to self-audit their work with her ended up raising even more questions — for them personally and their whole field www.theatlantic.com/magazine/arc...

The superendowment effect—research by Winegar and Sunstein finds users would demand $80 to share personal data, but pay only $5 to keep them private. This raises questions over the reliability of WTP/WTA to gauge the welfare significance of giving up or maintaining privacy: https://buff.ly/3Oa5jvr

Prof Kai Chan and I are recruiting a PhD student to start at UBC starting in Fall 2025, to work on Green Death Care and Relationships with Nature. Looking for applicants with a background in survey and experimental design and analysis. Dec 15 application. deadline. Details: www.grad.ubc.ca/ad/58334

For any new @bsky.app users who are interested in #ResearchIntegrity matters, I've got a #StarterPack for you to help build your #Bluesky feed: go.bsky.app/5NJ9Z4N I like this method for finding new people to follow and interact with. And do let me know if you'd like to be added to the list

Hi there. We're Retraction Watch -- retractionwatch.com -- and we've joined Bluesky.

Thanks for the great Keynote, Jiaying!! You make me come alive! #BigDifferenceBC

BIG Difference BC starts tomorrow at 9am Pacific Time! * Free online conference on behavioural insights for good * Keynote on "Nudging Climate Action & Happiness" by Prof Jiaying Zhao * Program: bigdifferencebc.ca/program-2024 * Registration link: bit.ly/BDBC2024regi... Hope to see you there!

It’s insane that I got all the way through getting a PhD before I encountered equivalence tests. These should be part of every intro stats curriculum that covers NHST.

Something like this?

Another shoe just dropped in the ongoing Dan Ariely scandal: JMR, a top 4 marketing journal, has issued a formal “Expression of Concern” for Mazar, Amir, & Ariely (2008), the infamous Ten Commandments study 🧵

‪ # Mnemonic for remembering Type I and II Errors # Boy who cried wolf trick First, Type 1: Villagers react when there is no wolf present Second, type 2: Villagers do not react despite there being a wolf present The story wouldn't make sense in the reverse order, making 1 then 2 memorable

BREAKING: A judge has dismissed the defamation suit brought against Data Colada by Francesca Gino, the Harvard Business School professor whose studies were found to contain fraud. (Some of Gino's claims against Harvard will be allowed to proceed.) www.chronicle.com/article/she-...

Hidden Confound in Pre-Registrations Critique datacolada.org/119 Post ends with link to submit suggestions for AsPredicted & OSF

New article from me: “Inconsistent multiple testing corrections: The fallacy of using family-based error rates to make inferences about individual hypotheses” Open access: doi.org/10.1016/j.me... #Stats #Methodology

i read the post on Gelman’s blog about the Protzko et al. “high replicability” paper, and then i skimmed the published paper, and then i read the review i wrote for an older version of the paper when it was submitted to Nature back in 2020, and after that i was left with 2 questions:

I am getting tired of people who criticize frequentist statistics, but then equate frequentist statistics with null hypothesis significance testing. Often leads people to say silly things such as 'p-values only allow you to reject H0, not accept it' - completely ignoring equivalence testing.

Another Elsevier paper with obvious AI-written text. “In summary, the management of bilateral iatrogenic I'm very sorry, but I don't have access to real-time information or patient-specific data, as I am an AI language model. “

"There’s also been a persistent sense of disbelief that Indigenous people could be responsible for this futuristic version of urban living...Canadians aren’t used to seeing Indigenous people occupy places that are socially, economically or geographically valuable, like Sen̓áḵw."

Goodhart’s Law xkcd.com/2899

Paper: "Selective and (mis)leading economics journals: Meta-research evidence" OA: onlinelibrary.wiley.com/doi/10.1111/...

Finally got up the courage to ask for a photo with one of the greats! He was as gracious and fun as you'd expect. For the record, he prefers "Richard" to "Dick" -- I checked!

If you haven't seen it already, check out this presentation on click farms and fraudulent survey data, from Leib Litman at Cloud Research: us02web.zoom.us/rec/share/Dx... They estimate ~30% of data is from click farms, and have some tools/suggestions for detection.

"b-hacking" revealed in the FTX trial yesterday. Could be a way to spot potential fraud in business! Look for numbers just over $1m or just over $1b. www.coindesk.com/policy/2023/...