Stretching DuckDB w/ Common Crawl, ~1.7B rows, ~300 parquet files. ~2-3s for single-column aggregations, ~2-3 mins to SUMMARIZE the data, peaking at ~12-14GB memory usage. Not exactly real-time, but the fact you can do this on a laptop with no server setups or Spark pipelines is still amazing.

Remember this #rstats post? I wasn't the only one talking about it & the tidyverse team was listening 😎 #databs New #dplyr functions? They're looking for feedback!! 🤔 replace_when, recode_values, replace_values 👀 Read this: github.com/tidyverse/ti... 🗣️ Comment on PR: github.com/tidyverse/ti...

I think pedocon theory is right. It’s empirically adequate, parsimonious, fits within a broader theoretical framework, and has immense explanatory breadth and depth www.liberalcurrents.com/we-need-to-t...

I am such a sucker for frivolous uses of AI. Here's an anthem for the tidyverse: suno.com/s/iVMVs4IoyA...

Very cool to see authors of this article mentioning the importance of sharing project-, data-, AND variable-level documentation alongside data in a repository, and linking to the templates I've provided on OSF as an example! 🌟 doi.org/10.1515/ling...

As a data manager, good documentation not only helps me do my job better, but also helps me annoy you less! 😅 Good documentation about inclusion criteria, READMEs about oddities in the data, consort diagrams and tracking to explain missing data, and so on, are all ways to ensure I bug you less! 🐛🐜🐝

New to me is the term "premature closure", where you too quickly latch on to the first solution you see. Always a danger in coding, but particularly so today when LLMs can give you a plausible fix so so quickly. www.shayon.dev/post/2025/16...

Bleeding edge update for the #tidyverse purrr package with even more seamless #rstats parallel maps. Introducing our shiniest new adverb: `in_parallel()`. Just wrap your function to take advantage of blazing fast parallel processing via mirai. pak::pak("tidyverse/purrr") purrr.tidyverse.org/dev/

One cool thing you can/should do is sample from priors only, and plot the distribution of the actual quantity of interest (ex: risk ratio). I find this very useful. This is actually super easy with brms. arelbundock.com/posts/margin...

This blog post about engineering not doing ETL is nine years old… it’s worth reviewing multithreaded.stitchfix.com/blog/2016/03...

Here's a functional programming trick for #rstats that I wish I started using sooner: if you need a #ggplot2 scale to be reusable across multiple plots and dynamically configurable without relying on global state, consider using a function factory (a function that returns a function) to build it

mirai - minimalist async framework for #RStats - released as an 'r-lib' package. Blog post: Advancing Async Computing in R. shikokuchuo.net/posts/26-mir... mirai provides event-driven async for #RShiny and parallel processing for purrr #tidyverse. Really excited to be working on this at Posit!

tl;dr — this EO co-opts the language of open science to implement a system of political control wherein presidential appointees are given broad latitude to designate any number of reasonable scientific activities and inferences as scientific misconduct, and to penalize those involved accordingly.

There's so much polarization around LLMs. They are way overhyped, I agree. But I also use them semi-regularly now. Here's a thread of genuine use cases where I find them helpful. Please add your own!

📦 I’m excited to share a new #rstats package I’ve been working on: {shinyfa} built to help folks working on large or unfamiliar #rshiny apps ✨ The package scans your app folders and extracts out details on render*(), reactive() and input$ to a dataframe! 📖 www.dalyanalytics.com/blog/shinyfa...

Playing around with satellite imagery of #madison to make some office art. #Rstats

✨Use llms from #rstats with ellmer ✨Version 0.2.0 is on CRAN now. No blog post yet because I'm about to go on vacation, but in the meantime you can check out the release notes: github.com/tidyverse/el....

The kind of Friday morning content I needed to see. ❤️

Registration for the posit::conf(2025) virtual experience is now open! Join us virtually, Sept 16–18, and access live-streamed keynotes and 100+ talks, on-demand recordings, Q&A sessions, and our virtual networking platform. Learn more in the blog post: posit.co/blog/posit-c... #RStats #Python

In case you missed it, we recently updated some of our packages, including many new features (again) in the #rstats #easystats {modelbased} package: easystats.github.io/modelbased/n... The last weeks we were working a lot on improving support and performance for Bayesian models and especially

I'm still thinking about my favorite quote from the Posit Data Science Hangout today. It perfectly sums up what I hope I provide to the researchers I work with: a trusted partner, who is there to support them in their work. Earn a reputation for being a good person to work with - Cara Thompson

Great news! R/Medicine 2025 is providing a forum for sharing R based tools and approaches used to analyze and gain insights from health data. Join us for the premier R conference for health and medicine. 🔗 Register today: rconsortium.github.io/RMedicine_we... #rstats #opensource #RMed25

I think a lot about what Carl Sagan said in one of his final interviews.

I’m happy to share that I’ll be giving a talk at R/Medicine 2025! 🎊 I work with a BIG REDcap database for substance use treatment (200+ locations) which makes extraction difficult. I developed {redquack}, an #rstats 📦 that transfers REDCap data to DuckDB, and will talk about how to use it. 🦆

📊 🕵️‍♂️ #rstats community! Do you sometimes feel like you're just pretending to be a data scientist? I'm researching imposter syndrome for my upcoming talk at posit::conf(2025) 🔍 I'd love to hear YOUR experiences in a short 5-10 minute anonymous survey: forms.gle/YkJtwZWquyKM... Please share! 🔄

Since it's in Atlanta, I'll be here at my first posit::conf! I'll be speaking here with @gosterhout.bsky.social about election night reporting with #rstats and #QuartoPub (showing off {targets} and other neat tricks like this: www.andrewheiss.com/blog/2024/11... )

States spend too much on clunky election night reporting. We just replaced ours using #rstats. dbplyr backend + reactable & leaflet viz + #quartopub site. Real magic happened with programmatic code chunks & targets pipeline done by @andrew.heiss.phd. #dataviz results.voteidaho.gov

Another q for the stats people! People worry about collinearity (cf blog post below). Consider a scenario in which the collinear predictors are just controls to account for confounding. Including both of them doesn't impair the precision with which the effect of interest is estimated, does it?

Happy reinstalling-all-your-R-packages day to all those who celebrate #rstats

Hive mind, please help me out! I need a more informative and explicit subtitle for my upcoming #RStats book "Model to Meaning" The premise is that analysts should often transform coefficient estimates into more meaningful / interpretable quantities like predictions, risk differences, slopes, etc.

My amazing independent study student wrote me a thank you card and drew this laptop with #rstats code on it 🥹

When planning for data collection, especially in longitudinal studies, first consider how that data will be used. Ask yourself: - How will we combine data for analysis? - What unique IDs will allow us to do this? - How will we name/code items to combine data? - Will our data need restructuring?

Recently I posted a draft of my essay on banishing p-values/NHST. I've now refined and expanded it, partly based on feedback I received here. Please take a look and comment, matloff.github.io/No-P-Values

If you're using LLMs to write R code, @simonpcouch.com's blog posts are the best way to keep up with which model is best

Childhood lead exposure can cause serious developmental delays, hearing loss, and behavioral problems. The CDC has long had the technology and the people power to efficiently and effectively mitigate lead exposure. RFK Jr. cut the program. What's more un-American than that? #StandUpForScience

use() is a pretty cool addition!

It’s a good day. I got my first suit and ran my first stan (brms) model in the same day. Except the full model is still running lol.

For the last couple of months I've been working on something and I'm excited to finally share an early preview: Say hello to plumber2 🎉 plumber2 is a full rewrite of the plumber package for creating powerful webapis in #rstats. It takes everything we have learned from plumber and adds even more

Visualizing git commits for ellmer using hellmer for batching with structured data refinement. 📦 #rstats gist.github.com/dylanpieper/...

today we will all read imbens 2021 on statistical significance and p values, which is a strong contender for having the best opening paragraph of any stats paper pubs.aeaweb.org/doi/pdf/10.1...

From Minnesota State Capitol today - #HandsOff @handle.invalid @indivisiblemnleg.bsky.social

Madison, Wisconsin, this afternoon: wow! #Handsoff #Protest (Photo via @captimes.com)

Madison, Wisconsin

Shot a bit on black and white today for the first time in five years and it felt good.

ADHD — Attention Deficit Hey Dude

this week in #rstats 🔥 - evaluating LLMs in R 🧪 @simonpcouch.com - Observable JS for R users 📊 @nrennie.bsky.social - a new Docker pkg for R 🐳 @coatless.bsky.social - using duckdb & duckplyr 🦆 @rorylawless.com - R pkg risk & QA 🚧 @jumpingrivers.com and more! 🥳 www.linkedin.com/pulse/week-r...

Your tariff rate is your star sign divided by the number of boys you've kissed, multiplied by one

Just did the math and yes the tariff rate is *exactly* (abs(trade_deficit))/us_imports for each country. Where this number is <10%, it's 10%. Literally every single country matches precisely.

📢 Huge Neb-RUG announcement! 📢 Yihui Xie will introduce his new litedown package in the next Nebraska R User Group talk: Creating Beautiful PDFs via HTML/CSS/JS 1-2pm CDT Wednesday, April 9th Register at www.meetup.com/neb-rug/even... Learn more about Yihui in the thread! 👇 (1/2) #RStats #statsky

We're looking at Pokemon data for #TidyTuesday this week! 📊 Stream plot showing speed and colour using {ggstream} 📷 Image added with {ggimage} 🎉 Aiming for a minimalist, fun, and arty chart Code: github.com/nrennie/tidy... #RStats #DataViz #ggplot2