Preprint "Reply to Westwood: Questioning empirical evidence that AI survey contamination is real and substantial" w/ @soubhikbarari.bsky.social, Trent Buskirk, @andrewgordon.bsky.social, Sunshine Hillygus We respond to claims of "existential threat" of AI on survey research osf.io/preprints/so...

Peer review reliability is shockingly low. Meta-analyses show reviewer agreement barely above chance, and grant outcomes often depend more on who reviews than what's proposed. Our new preprint with Agnes Urban and Arjun Krishnan @compbiologist.bsky.social : papers.ssrn.com/sol3/papers.... 🧵 1/

The instructions are occasionally outrageous, and sometimes the barks sound like human speech. But a fully autonomous, free-thinking, conversational dog park, Moltbook is not.

Is the "AI takeover" a looming reality or a clever illusion created by human steering? @soubhikbarari.bsky.social, a Senior Research Scientist at @norc.org’s SDC, breaks down why the "Moltbook" phenomenon is less about sentient agents and more about "dogs with walkie-talkies." #Moltbook 🧵

Interesting experiment in patient diagnosis using LLMs highlighting the critical differences between LLM output when used optimally, and LLMs as actually used by real people www.nature.com/articles/s41... Key takeaways:

Federal agencies often work with data not originally designed for statistical use. NORC researchers developed a framework for how AI, with human oversight, can improve the quality of that data for policy decisions. Find out more. go.norc.org/4aexK5T

*Tap* (is this thing on?) In the event that anyone's still out there, some new writing from me. I looked at country-level AI usage from Anthropic’s Economic Index and asked a simple question: beyond GDP, what cultural values track AI adoption?

Thought provoking post from @soubhikbarari.bsky.social on uses of LLMs in social sciences. Screenshotting the section most relevant for surveys. somewhatunlikely.substack.com/p/100-ideas-...

Interesting (to me, as native Wisconsinite) post by @soubhikbarari.bsky.social on who identifies as Midwestern somewhatunlikely.substack.com/p/the-midwest

College rankings wield enormous influence, but do they actually measure quality? NORC research methodologist @soubhikbarari.bsky.social reveals critical flaws and solutions for improvement to the college ranking system in his latest Expert View. Find out more: go.norc.org/46GIx8l

Reality TV is civic education. My review of @eunjikim.bsky.social's excellent new book “The American Mirage: How Reality TV Upholds the Myth of Meritocracy" up now. open.substack.com/pub/somewhat...

Fun facts: though everyone knows Zohran is the media darling of the NYC mayor race, Lander's arrest last week skyrocketed his mentions in the national media, more than any other candidate on any day this year. (Also Cuomo has clearly been the more consistently salient candidate in print/broadcast.)

How the public feels about AI. My latest on Substack: a short but sweet round-up of some public opinion studies over the past ~7 years on AI. Some surprising patterns, some not so surprising, and a lot we don't know. Link below 👇

This week @aapor.bsky.social, I'm excited for my @norc.org methodology colleagues to present (and debut!) some exciting work at the intersection of #ai and #surveys. A quick round-up 🧵 ⤵️

This week @aapor.bsky.social, I'm excited for my @norc.org methodology colleagues to present (and debut!) some exciting work at the intersection of #ai and #surveys. A quick round-up 🧵 ⤵️

Another survey methods talk at 𝗡𝗬𝗔𝗔𝗣𝗢𝗥 coming up next week to put on your calendar: Wed April 23 - 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗔𝗜 𝗳𝗼𝗿 𝗦𝘂𝗿𝘃𝗲𝘆 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 (led by @yamilrvelez.bsky.social , @joshua-lerner.bsky.social , and myself) RSVP below 👇

Another survey methods talk at 𝗡𝗬𝗔𝗔𝗣𝗢𝗥 coming up next week to put on your calendar: Wed April 23 - 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗔𝗜 𝗳𝗼𝗿 𝗦𝘂𝗿𝘃𝗲𝘆 𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵 (led by @yamilrvelez.bsky.social , @joshua-lerner.bsky.social , and myself) RSVP below 👇

How can we redesign college ranking systems to better serve students and advisors in their college search? Check out this upcoming webinar for insights from NORC's @susanpaddock.bsky.social and @soubhikbarari.bsky.social. Learn more and register: go.norc.org/41S7sn5

New in 𝗧𝗼𝗯𝗮𝗰𝗰𝗼 𝗖𝗼𝗻𝘁𝗿𝗼𝗹 w/my @sdc.norc.org colleagues @kostygina.bsky.social, Chandler Carter, @andarist.bsky.social, and @sherryemery.bsky.social: on political appeals in nicotine pouch ads. (Which, as "Torches of Freedom", reminds us, is not new). PDF version: soubhikbarari.com/writings/tuc...

Can AI-powered textbots improve survey interviews? Join PRIISM and @soubhikbarari.bsky.social to examine how they can enhance response quality and scalability, while considering challenges in bias and respondent experience. If you're in NYC, join us in person! steinhardt.nyu.edu/events/ai-as...

When gauging public reactions to the latest controversy, it's important not to conduct or share misleading polls. Our new paper in Public Opinion Quarterly is a template for how to (and how not to) poll about the effect of political developments. Gift link: academic.oup.com/poq/advance-...

🚨 Upcoming Webinar Alert 🚨 Check out this webinar from NORC's @soubhikbarari.bsky.social and @joshua-lerner.bsky.social on using NLP and generative AI in survey research! RSVP for the webinar on February 26! wapor.org/resources/wa...

Some folks asked me how to build a data science career as a social scientist. I published an article on this with other computational social scientists last year: cambridge.org/core/journal...

Coverage of some results from a population segmentation study we conducted ahead of the election that revealed pervasive cynicism across almost all groups of Americans. Always a joy to collaborate with @trosenstiel.bsky.social, @soubhikbarari.bsky.social, and colleagues.

Can AI-powered survey interviews boost response quality? Our study shows conversational LLMs elicit richer, more detailed responses vs standard formats. Read the full brief from NORC staff on enhancing survey methodology with GenAI: go.norc.org/3V2G81D

this AP-NORC poll on Thanksgiving foods is from last year, but I simply cannot stress enough how much I continue to enjoy the phrase "Casserole unspecified/other" apnorc.org/projects/tur... via apnews.com/article/ap-p...

Thanks @mirya.bsky.social! See descriptives on when / where nationally salient partisan language appears in local politics (“climate change,” “gun control,” etc.). Identifiably partisan language is (1) common, (2) especially in larger cities, and (3) local D / R “slant” correlates with pres. vote.

🚨New paper out in 𝐏𝐮𝐛𝐥𝐢𝐮𝐬🚨 This one was a labor of love with @simko.bsky.social synthesizing many of our thoughts (and work) on the intersection of computational social science and local political economy over the past few years. Thanks @mirya.bsky.social for the bump!

Excited to share a first look at our AI-in-survey research from this summer! @norc.org

Still digging into the election? Tune in to [email protected]'s roundtable on Nov 21. We have a great lineup from the networks (@kabirkhanna.bsky.social /CBS, Arnon Mishkin/Fox, @trussler.bsky.social/NBC), polling (@pollsterpatrick.bsky.social), and academia (@michaelsb.bsky.social). RSVP↓

Still digging into the election? Tune in to [email protected]'s roundtable on Nov 21. We have a great lineup from the networks (@kabirkhanna.bsky.social /CBS, Arnon Mishkin/Fox, @trussler.bsky.social/NBC), polling (@pollsterpatrick.bsky.social), and academia (@michaelsb.bsky.social). RSVP↓

This is exactly @eunjikim.bsky.social's research area. Reality TV shapes views about the American Dream: onlinelibrary.wiley.com/doi/10.1111/... Police procedurals likely influence attitudes about the carceral state (work in progress) Entertainment TV has under-appreciated effects.

Vanderbilt University asked NORC to examine college ranking systems and their methodologies. Find out what we learned and get the details on our work with Cosmic Explorer in the latest NORC Now.

I'm excited to be hosting (NY)@aapor.bsky.social's virtal 🗳️pre-election🗳️ roundtable on October 28th. We're joined by an illustrious group of experts: @ryanenos.bsky.social, @nataliej.bsky.social, @karthick.com, and Ruth Igielnik. Don't miss out! (RSVP 👇)

Excited to kick off NYAAPOR's fall series on public opinion/elections/data science. In two weeks, Professor @eunjikim.bsky.social will lead an in-person discussion of her amazing POQ using Yelp reviews to measure anti-Chinese bias during the pandemic. NYC-area folks, RSVP in the reply!

Excited to kick off NYAAPOR's fall series on public opinion/elections/data science. In two weeks, Professor @eunjikim.bsky.social will lead an in-person discussion of her amazing POQ using Yelp reviews to measure anti-Chinese bias during the pandemic. NYC-area folks, RSVP in the reply!

NYC #survey researchers, political scientists, and #polling enthusiasts: Don't miss NYAAPOR's upcoming paper talk/discussion/happy hour this coming Tuesday at 6:30 in Midtown 🍻. Featuring Jason Barabas on the importance of bridging polling and #democracy. Register here: tinyurl.com/nyaapor-poq

I’m excited to say my study of political signals from corporate brands on social media is now published in a special issue of the Journal of Quantitative Description! doi.org/10.51685/jqd... I’ll be presenting @ ICWSM on Tuesday at 11:30, so drop by if you’re around ⚡️.

I’m excited to say my study of political signals from corporate brands on social media is now published in a special issue of the Journal of Quantitative Description! doi.org/10.51685/jqd... I’ll be presenting @ ICWSM on Tuesday at 11:30, so drop by if you’re around ⚡️.

I cannot stress this enough, right now if you see a dramatic or alarming headline about young people based on a poll, you should be immediately suspicious. The problem is specifically with online opt-in polls, but these are the ones that most frequently produce these crazy results that go viral.

Another banger from NYAAPOR for #NYC #survey folks: join us on Tuesday June 4th after work for an interactive workshop on #qualitative methods! Register here: www.eventbrite.com/e/crafting-a...

If you're in town for @aapor.bsky.social #aapor2024 and you're a nerd for complex survey weights, stop by 218 at 2pm on Friday where I'll be presenting some on-going work between @norc.org and #NCHS on combining prob and non-prob surveys 📊

Survey folks: any good resources / advice out there on the design of 'usability' questions (beyond the generic survey satisfaction q)? Much appreciated!

Poli sci/survey/UX folks in the NYC area: NYAAPOR and the Insights Association is hosting a *free* happy hour this Monday ahead of @aapor.bsky.social conf next week. If you're cool, we'll see you there: 😎🥂 tinyurl.com/nyaapor-drinks

"it’s a mistake to argue with a book that contains no arguments."

How many Americans do you think eat pizza every single day? A recent Netflix show committed a statistical felony in answering this question. It's a statistical incrustice, leaving mushroom for improvement. My latest Substack: somewhatunlikely.substack.com/p/alotta-pizza

Olive for this kind of endorsement.

How many Americans do you think eat pizza every single day? A recent Netflix show committed a statistical felony in answering this question. It's a statistical incrustice, leaving mushroom for improvement. My latest Substack: somewhatunlikely.substack.com/p/alotta-pizza

I read a lot of great social science in 2023. I decided to write about 3 books, in particular, that say something very important about the state of American Politics this ̶c̶o̶m̶i̶n̶g̶ ̶ current(!) election year: somewhatunlikely.substack.com/p/best-reads...