On Thursday I'll be taking part in a roundtable “How real is the LLM threat to online research in academia?” for @joinprolific.bsky.social alongside @davmicrot.bsky.social, Michael Nicholas Stagnaro, and Raluca Rilla. Sign up here: lnkd.in/em8-MpjN

📢 Webinar series: How to bot-proof your online research In the first of our three-part series, experts from @msftresearch.bsky.social, MIT / Stanford, and @mpib-berlin.bsky.social unpack the threat of LLM-powered bots and AI agents to research integrity in academia. #AcademicSky #Research

See you in Chicago! If you're heading to the @spspnews.bsky.social Annual Convention from Feb 26-28th, stop by booth #19 to find out how you can use Prolific’s new bot detection tool in your next study. 🎁 $50 credits offer available for first-timers. #AcademicSky #SocialPsyc #Research

On the first episode of The Frontier, we spoke to Crystal Qian, Senior Research Scientist at Google DeepMind about Deliberate Lab: an open-source platform for behavioral human-AI social experiments. 🧪 Developed by the PAIR (People + AI Research) team. #MLSky #AcademicSky #ArtificialIntelligence

AI agents are becoming a serious threat to research data quality. Today we’re rolling out Bot authenticity checks on @joinprolific.bsky.social, detecting agentic AI with 100% accuracy in testing. Comes with a native Qualtrics integration! More info: www.prolific.com/resources/in...

Introducing bot authenticity checks that detect agentic AI and bots in academic research. 100% accuracy in testing. Our tool flags automated browser scripts or non-human behavior within your Qualtrics studies, so you can collect data with high confidence that it's human. #AcademicSky #Research

What drives retention in online longitudinal research? We conducted an experiment (N=1,798) on @joinprolific.bsky.social, orthogonally manipulating payment rate (£6–£9/hr + bonus) and session interval (1, 2, or 4 weeks) across five sessions. The findings challenge some common assumptions 👇

When platforms consolidate, research options shrink. Researchers are rightly asking whether participant access will stay tool-agnostic, high-quality, and reliable. As @phe-lim.bsky.social says, we believe access works best when it’s open by design. So if you're reassessing, get in touch. #Research

AI modelling, product research, and academic studies all hinge on one thing most platforms can’t guarantee: real, trustworthy human data. 👥 This is where @joinprolific.bsky.social comes in, the platform-model tech company filling a critical gap in AI and research by providing fast,... (1/11)

Thank you 2025. You’ve been great. ⭐️ Together, our global community across AI, research, and participants made this a truly monumental year. Whether you were collecting high-quality data or contributing your expertise, we were honored to be your human data platform of choice.

AI agents are becoming more sophisticated, more capable of mimicking human behavior, more accessible. This raises questions about how we keep real human feedback at the core of online research. 🧵 #AcademicSky #ResearchIntegrity

🇬🇧 At the @universities-uk.bsky.social conference, Dr. Simon Jones discussed digital transformation in research services, with insights from 35,000+ researchers using Prolific. He explored challenges and successful institutional practices. Great to sponsor another #UUK! #AcademicSky #Research

Lots of chatter about this paper currently. Its a stark warning, but at present I see this as a stark warning of what might come, not what is happening now. As a research community we need to see it as a call-to-arms to develop new strategies, NOT a call to abandon online sampling. Reasoning below

Without minimising the seriousness of the threat raised in this paper, I'm more optimistic. This is just the latest challenge in online integrity of online research. We've been proactively adding to our suite of authenticity tools - more every week - including many of Sean's recommendations:

We’re proud to surpass multiple industry benchmarks in the Insights Association’s recent Global Data Quality (GDQ) Benchmarking Report for online #research data quality. Our metrics include: ⭐️ 2.7% pre-survey removal rate ⭐️ 0.5% in-survey behavior removal ⭐️ 0.5% post-survey cleanout rate

There's not just a generational wealth gap anymore. There's a wellbeing gap too. And right now, it's hitting younger people the hardest. (Data from our latest @joinprolific.bsky.social poll - November 2024) Full data: www.prolific.com/polling

Protocol is our five-layer system built to ensure your data is of the highest quality. VP of Product Sara Saab guides you through the rigorous identity and quality checks, in-study detection tools, and more that ensure your research is powered by human intelligence, always. #AcademicSky #Research

In an AI-powered world, genuine human perspectives are both precious and vulnerable. The line between human and AI-generated responses is blurring, threatening the very foundation of academic research. Our approach to protecting data integrity 🧵 #AcademicSky #Research

🎓 Great news for academic researchers - our $100 / £100 free credit offer has been extended! Sign up with your university email to collect fast, high-quality data from 200K+ vetted participants across the globe. 👉 www.prolific.com/academic-res... #AcademicSky #PhDSky #AcademicResearch

What is one question you'd dearly love to ask a representative sample of 2,000 US adults? Going to try this again because it was super interesting last time. I'll stick the best question in our next @joinprolific.bsky.social poll that goes live at the start of November. #polling #research

Limited time offer for academic researchers 🚨 Get $100 / £100 free credit this October to run your next study with fast, high-quality data from 200K+ verified participants on Prolific. Sign up and redeem: www.prolific.com/academic-res... #AcademicSky #PhDSky #Researchers

The Double Squeeze hitting Trump's base. 9 months of @joinprolific.bsky.social tracking show voters report surging costs AND growing fears of US global decline. They voted for prosperity & strength. They got strain & decline. Full analysis: www.prolific.com/resources/tr... #poll #geopolitics

How to design studies and get quality data with Dr. Simon Jones 🎥 Join our Senior Research Consultant's refresher as he shares expert tips to optimize data quality, and runs through the latest platform features designed to make launching studies faster and easier than ever. #AcademicSky #Research

📢 Authenticity checks are out of beta, with upgrades for ease of use. Authenticity checks work with Qualtrics and Prolific to detect AI-generated responses in your research with 98.7% precision, so you can trust the data you collect is authentic. #AcademicSky #ResearchIntegrity #AIDetection

AI benchmarks show models like GPT-5 might ace scientific reasoning, or Gemini and Claude adapt to new concepts. But which LLMs offer the best user experience? @tomsguide.com explains how Prolific's AI leaderboard HUMAINE is a big step forward. #ArtificialIntelligence #MachineLearning #LLM

Super excited to share this work with the world! We wanted to put human preference at the heart of our AI model leaderboard, and to do so in a principled and representative manner. The result: HUMAINE Check out the data here and get in touch with any questions: huggingface.co/spaces/Proli...

Introducing HUMAINE: the LLM benchmark that puts real human experience first 🎯 21,352 human evaluators. 27 models. 22 demographic groups. 5 evaluation dimensions. In partnership with @hf.co. See insights: huggingface.co/spaces/Proli... More in 🧵 #ArtificialIntelligence #MLSky #LLM #Developer

New: Periodic identity confirmation checks with video recording to ensure authentic, human feedback in your academic and industry research. Along with bank-grade ID verification, this ensures participants are who they say they are—not bots, duplicates, or fraud. #AcademicSky #ResearchIntegrity

AI/ML researchers in San Francisco, we have a few spaces left at our community event 🇺🇸 Join us Sep 11 for this month’s theme: Researchers building real-world AI tooling. With speakers Prolific CEO Phelim Bradley and Jiaxin Pei, Postdoc at @stanfordhai.bsky.social. 👇 #MLSky #ArtificialIntelligence

🔍 Audience Finder is here. Instantly check if the participants you need for your research are on Prolific. No set-up required. Just type who you’re looking for (“Bilingual psychologists who speak Mandarin”) and see live results in seconds. ➡️ www.prolific.com/audience-fin... | #AcademicSky

In the age of AI, authenticity matters more than ever in academic research. LLM-generated responses, false demographics, and more pose a threat. VP of Product Sara details tools and internal measures we’ve implemented to help maintain research integrity 🧵 #AcademicSky #ResearchIntegrity

"Through three experiments (N = 975), we found that bots, though more prosocial than humans and linguistically distinguishable, were not selected preferentially when their identity was hidden. Instead, humans misattributed bots’ behavior to humans and vice versa." arxiv.org/abs/2507.13524

New preprint -- we (me, @klempert.bsky.social, Dave Wolk, and Joe Kable) examined whether characteristic aging-related changes in cognition and personality show up on six online platforms (3 crowdsourcing sites -- MTurk, CloudResearch Toolkit, & Prolific, and 3 panels). osf.io/preprints/ps... 1/

Sharing a piece I wrote for The AI Journal on who should govern AI: aijourn.com/a-fine-balan... Our recent @joinprolific.bsky.social polling shows 69.7% of people think AI investment will primarily benefit corporations not the public. They're probably right. [1/3] #AI #AIGovernance #TechPolicy

The largest investigation of AI persuasion with 76,977 participants across 3 large-scale experiments. Excellent work by @ox.ac.uk PhD candidate @kobihackenburg.bsky.social. 19 LLMs. 707 political issues. 466,769 fact-checkable claims evaluated. arxiv.org/abs/2507.13919 #AcademicSky #MLSky #PhDSky

4 new security enhancements for better data quality in your academic research 🧵 With data quality an absolute priority yet challenge for many researchers, we’ve made several important updates to strengthen security and maintain data quality on the Prolific platform. #AcademicSky #PhDSky #Research

🎥 Webinar: Why AI leaderboards miss the mark 🚫 @cohereforai.bsky.social’s Oliver Nan, University of Washington's @huashen.bsky.social, and Prolific’s Nora Petrova will be discussing where leaderboards are falling short, how to improve them, and more. 🔗 www.prolific.com/resources/wh... | #MLSky #AI

Pleased to have fueled some of the most fascinating research in recent years. With features from Google DeepMind, Microsoft, MIT, and more, you can now explore impactful research and AI alignment stories empowered by Prolific's diverse participant pool. 🔗 www.prolific.com/resources?ty...

Meet Protocol—Prolific’s proprietary data protection system. For years, Protocol has been pivotal to maintaining high data quality standards on our platform through automated, continuous monitoring. Here are 3 things to know ⬇️

“We don’t compete with our customers. We don’t have conflicting incentives." @businessinsider.com reports that Meta’s 49% stake in Scale AI has competitors Google, OpenAI, and xAI turning to other data providers. Prolific CEO Phelim Bradley reflects on the impact. #MLSky #News

When you chat with an LLM, doesn’t it sometimes feel like you’re talking to another human? Here's how 1,000+ Prolific taskers helped @ox.ac.uk's @oii.ox.ac.uk assess their evaluation method that could identify human-like behaviors among LLMs 🧵

Our San Francisco hub is officially open! 🌁 Now coast to coast, we’re proud to be where our AI partners innovate, scale, and lead.

Take a tour of authenticity check, the new AI detection tool for research. You get to hear how this new feature (beta) helps you maintain the genuine human feedback that your academic research relies on, along with the best tips for the highest-quality data. #AcademicSky 📺 youtu.be/TXoK26-f0f0

What do domain experts really want from contributing to AI/ML? 💬 We surveyed 100+ Domain Experts on why they take AI studies, what they’ve learned, how it affects their own perception of AI, and more. #MLSky

⚔️ Machine Learning Street Talk (MLST) exposes the problem with Chatbot Arena as LLM benchmarks neglect user preference. AI builders are battling for the top spot. But to build those models with precision and depth, high-quality human data is critical. Big things coming with MLST! #MLSky

LLMs learn beliefs and values from human data, influence our opinions, and then reabsorb those influenced beliefs, feeding them back to users again and again. We call this the "Lock-In Hypothesis" and develop theory, simulations, and empirics to test it in our latest ICML paper!

📈 Prolific recently featured in the @odihq.bsky.social (ODI) report “Mapping the Role of Data Work in AI Supply Chains,” which outlines the evolving landscape of work behind #AI data. Interesting read for developers and researchers implementing human-in-the-loop: theodi.cdn.ngo/media/docume... 🗞️

A night to remember at the Oxford AI Gala 🪉 Over the weekend we attended and exhibited at the Oxford AI Gala in @morethanadodo.bsky.social—sponsored by @anthropic.com. We also had the honor of demoing our human-centric AI leaderboard and connecting with fellow industry leaders. #AcademicSky

🌈 At Prolific, we believe true innovation is driven by true representation. Diversity, equity, and inclusion is not only something we champion in our team culture, but in the data our customers collect to help change the world. We hope you have a safe and happy #PrideMonth.

The rise of LLMs doesn’t have to mean the fall of real research. Our VP of Product Sara Saab says now that humans and AI working together is the new normal, we’re providing guidance on the use of #AI tools to ensure integrity. 🎥 Watch the full webinar: youtu.be/MBo50M6etCk