And this is why meta analysis is screwed

Sorry I gave them so much hell on this one but happy to see some solid discussion about the necessity of qualitative approaches. Bravo.

Our lab has three RRs of simulations underway. It’s a shame to tell my students that our top journal will not consider them for publication. Let’s change that!

Help us show there is support for offering registered reports at one of Psychology's leading method's journal! chng.it/TwwnVBScVb

"Preregistration" lumps three distinct practices into one: - Study registration - Protocol - Analysis plans (SAPs) To advance open science, it's critical that we distinguish them. 🌟 🌟 Now in-press paper from Evan Mayo-Wilson, @seangrant.bsky.social, David Moher, me: osf.io/preprints/me...

AMPPS Call for Papers: Replicability and Reproducibility in Methodological Research. Proposals due September 15. @jkflake.bsky.social 

Folks, it's time to submit your papers to PCI Psychology. Here is our invitation to you, taken from our editorial osf.io/preprints/ps...

Check out this awesome new paper about #OpenScience in developing countries! Paper is forthcoming at AMPPS- very proud that we can promote this work. Congrats to the large and international authorship team, and esp @hcp4715.bsky.social for leading the charge. @psychscience.bsky.social

A new preregistration template for simulation studies is now live on OSF. Thanks to the creators and the preregistration template working group for proposing and shepherding this to make it available to users. See the post if you would like to propose a new template. www.cos.io/blog/introdu...

Experimentology is out today!!! A group of us wrote a free online textbook for experimental methods, available at experimentology.io - the idea was to integrate open science into all aspects of the experimental workflow from planning to design, analysis, and writing.

🚨 Calling all dog behavior & cognition researchers - ManyDogs Project is launching ManyDogs 2! 🔍🐕‍🦺We're studying overimitation: Do dogs copy irrelevant actions just b/c their favorite human does them? Email [email protected] to collaborate globally and contribute to reproducible science!

I'm very happy to see meta-science on methods research getting more attention in the form of a great new call for papers! I recently gave some talks on the existing literature on the quality and transparency of sim studies & ideas for future improvement. Slides: bsiepe.github.io/talks/2025-0...

Are you working on issues of replication and reproducibility of methodological research in psychology? Consider submitting a proposal to my call for papers, happy to chat about your ideas or answer any questions! www.psychologicalscience.org/publications...

🎉 After 2 years of work, our #OpenScience guide for researchers in developing countries is LIVE! Check out our new preprint: osf.io/preprints/ps... We map the opportunities 🌟 + barriers 🧱 facing Global South researchers. *(1/4)*

PCI Psychology is here!! 🎉🥳 After over a year of hard work by so many people, we are thrilled to announce that we are open for submissions! Join us in making publishing more efficient, equitable, and open: psych.peercommunityin.org #PsychSciSky #scipub

Robustness analysis is going to be a big part of the future of how we think about evidence synthesis and rigor - exciting new journal where you can turn your blog posts of other ways the analyses could have been done into papers! Excited to see some submissions come in!

In the workshop we'll follow an example of developing a survey to measuring pickle fanatism for marketing purposes to learn about how to develop and evaluate survey questions for a variety of purposes. The workshop spans qualitative and quantitative methods. Two days of measurement fun!

HELP: Many of you will be familiar with the dataset that is often used to teach multilevel modeling with variable such as "MATHACH" and "SES" -- these data are used in papers, textbooks, and software packages. You may have the dataset on your computer, BUT, is it public use data?

Chuffed and excited to announce a new journal section in Cortex: *Methods and Assumptions* is open for submissions starting today! Read the opening editorial here (OA): www.sciencedirect.com/science/arti... 1/

Attention metascience & AI enthusiasts: SSHRC is offering $140K/2yr postdoc fellowships, and I'd be happy to sponsor someone to work in my lab at UofT. If you're interested in how AI might transform scientific practice (for better or worse), apply by April 8. Let's question everything together.

Please share! Not 100% official yet, but I may get to recruit a Ph.D. student for Fall 2025. If you are interested in psychological methods, math, and/or some combination of those, send me a message so we can chat!

I have to fix it in my head. This survey validates me.

I’ve long used FiveThirtyEight’s interactive “Hack Your Way To Scientific Glory” to illustrate the idea of p-hacking when I teach statistics. But ABC/Disney killed the site earlier this month :( So I made my own with #rstats and Observable and #QuartoPub ! stats.andrewheiss.com/hack-your-way/

Hope to see more papers like this that question the performance and trade-offs of more complicated and newer methods with a sample of datasets (not just simulations that can benefit from cherry picking). Combats the publication bias towards new methods and demonstrates on-going comparative work.

I'm hiring! I'm looking for 1️⃣ pre-doc and 2️⃣ post-docs. If you're fascinated by media effects, quantitative methods, and open science, please consider applying 🙂 International applications welcome! Application Pre-doc: jobs.uzh.ch/job-vacancie... Application Post-doc: jobs.uzh.ch/job-vacancie...

👀

HelioCampus is hiring! We have openings for data scientists and data analysts. www.heliocampus.com/company/care... I'm the hiring manager for the data scientist role, and we're specifically looking for someone with experience working with student data in the higher ed industry.

The methods and statistics group at @tilburg-university.bsky.social get it.

The looks great: “cross-cultural developmental research poses distinct problems for ensuring high construct validity, owing to the nuances of working with children; the standard approach of transporting protocols designed & validated in one population to another risks low construct validity”

😱 'The sheer number of EFA structures that was deemed 'passing' by CFA fit indices made me realize that perfect simple structure is really more of an assumption than a real tested proposition in CFA.'

Where is the hand brake? Hand brake hand schmake @eikofried.bsky.social

Some stuff I learned while visiting the @psicostat.bsky.social group at the University of Padova. 1. Super old, first psychology department in Italy and one of the first universities in the world 2. Galileo was faculty there and likely lectured from this podium

Learning about big team science projects with @debruine.bsky.social on day 3 at @psicostat.bsky.social winter school. Good chat about less sexy aspects of working in diverse groups: differences in ethics approval and date formats. Put the year first! 2025-02-19

We submitted to Frontiers?! 😱 It's a comment on a Frontiers paper (doi.org/10.3389/fpsy...) purporting to measure “problem gambling.” But they didn't! The scale was edited: “gambling” was changed to “spending money on gacha.” So it no longer measured “problem gambling.” 🤦 doi.org/10.31234/osf...

I didn’t realize I needed a thread with conference highlights, much better than an email or remembering to check a website. So excited to see everyone in Minnesota this summer!

Great history lesson from @lakens.bsky.social with the @psicostat.bsky.social group at the University of Padova. History is repeating itself, but we also get a little better after each crisis was the positive take away for me.

Visual Analogue Scales are often used to measure short-term changes in body dissatisfaction. But how much change (out of 101 points) represents a meaningful change? 20 points? 10? 5? In our new (open access) paper, we try to find out: www.sciencedirect.com/science/arti...

I'll be at Social and Behavioural Data Science Center at the University of Amsterdam for a couple of weeks. I'm giving a lecture on Monday Feb 24th that anyone is invited to and hope to meet up with some people while I'm in town to talk methods and measurement. sites.google.com/view/mellenb...

Re sabbaticals: how do you handle your lab/on-going research? Do you cut out or scale back? Do you work less or does research just fill the time that was teaching/service? If you travel/do research visits/try to learn things, how do you also run your lab? Do you tell people you'll be slower?

I just want to say when I called my lab, "flake lab" it was because I couldn't think of a name for it, and well it is my lab. But now it is routinely referred to as "flake lab" -- I'm sorry for all of those involved. Do we study flakes? No, we do not.

We've had a paper reviewed by ERROR, which has concluded we had a "major error that affects a core conclusion" Few comments on the conclusion and the process in general 1/n

It is an honor for anyone to care enough about your work to check it, so thanks dudes. A mistake I suggest others not make is failing to ensure that all the data, from download, to merge, to analysis is available and each step, not just the final analysis, is reproducible.

My first paper as an Assistant Professor at UC Davis is out in the journal SEM! In the paper, we introduce a continuous-time extension to the GIMME model implemented in OpenMx. It uses iterative tests of modification indices to construct group- and person-specific dynamic networks!

I know this is the nice place, but I'm on lists that anyone can see called things like, "harassers", "piss drinkers", "bigoted freaks", and "hate enablers." This appears to be based on accounts I follow or posts I like? Don't recall liking any piss refreshments. Seems like a toxic feature ☹️

I’ve received my fair share of bad reviews. I’ve probably written my fair share of bad reviews too. But, on the whole, my final works read better because people volunteered their time to critique my writing, my thinking, and my computational skills.

The significant mistake I made is waiting until December to invite reviewers on papers. Anyone super keen to review and have expertise in causal inference, mediation, or quant psych stuff in general?

What are your favorite papers that report on experiments designed to understand how people fake responses to surveys? Particularly interested in faking of personality scales, but also in other psychological constructs.

We have a call open for editor for Psychological Bulletin. Deadline is Monday January 13, 2025. If you know someone or you are interested see the link below. Its a great journal to be involved with. www.apa.org/pubs/journal...

Moral psychology postdoc opportunity at the University of Toronto! Please repost, and pass on to anyone you think might be interested.