When you collect data online, are the results from humans or AI? In a project led by Booth PhD student Grace Zhang, we estimate the prevalence of AI agents on commonly used survey platforms:
osf.io/preprints/ps...
🧵
Deliberate Lab is one framework already leading the charge. The technical paper is available here → arxiv.org/abs/2510.13011
Google DeepMind's Crystal Qian raises that while most RLHF evaluation pipelines remain static and single-user, models are increasingly deployed in dynamic, multi-party settings.
She underlines human-centric augmentation evaluations.
Discussion → youtu.be/Mk9EFnXfBxA | #AcademicSky #MLSky #Research
I've developed a psychological countermeasure for AI voice deception… and it's surprisingly simple! 🗣️
Thread below 🧵
doi.org/10.1093/cybs...
I see. Thank you for the feedback - the lock is in place due to certain data policies but your point is entirely understandable. It's been passed on to our product team to consider, and to potentially chat with you more about this in future if you're open to it. 🙂
Generally you can always appeal to our support team for a manual review, but if it fails manual checks this unfortunately means your account violates our participant terms and would remain on hold. Happy to look into it for you if you have the ticket ID number to hand.
Hi Merideth, you can actually select 15 filters twice before the selection locks, so you can capture 30 prescreener fields + 19 basic fields. This is to prevent an excessively large amount of data being exported at once. If you want to provide a study ID however we're happy to look into it further.
Come find us at #SPSP2026 Chicago today 👋
Speak to the team at booth #19 and discover how to use Prolific’s bot detection tools in your academic research. $50 credits offer available for new users!
@spspnews.bsky.social | #AcademicSky #SocialPsyc #Research
Session time: 5:00 PM GMT / 12:00 PM EST / 9:00 AM PST.
🔴 Join us LIVE: riverside.com/webinar/dire...
🔔 “How real is the LLM threat to online research in academia?” will be live today.
Experts from Microsoft Research, MIT / Stanford, Max Planck Institute, and Prolific discuss the threat of agentic AI to online research, and how to protect against it.
Link to join live below. #AcademicSky #Research
Hey, sorry to hear that! Account holds can be due to any mismatching information (e.g. PayPal name being different) and VPN errors can still be triggered if there are security measures you don't have oversight on, such as using wifi at work. In any case, our support team will review it for you.
We ran a controlled study of 125 verified humans vs 5 AI agents. Can agents reliably be detected?
Here's what we found:
www.prolific.com/resources/au...
On Thursday I'll be taking part in a roundtable “How real is the LLM threat to online research in academia?” for @joinprolific.bsky.social alongside @davmicrot.bsky.social, Michael Nicholas Stagnaro, and Raluca Rilla.
Sign up here: lnkd.in/em8-MpjN
Coming up next:
🎥 Live demo: Authenticity checks detect agentic AI with 100% accuracy
🎥 How to verify participants on Prolific
Pick your sessions to register → resources.prolific.com/how-to-bot-p...
📢 Webinar series: How to bot-proof your online research
In the first of our three-part series, experts from @msftresearch.bsky.social, MIT / Stanford, and @mpib-berlin.bsky.social unpack the threat of LLM-powered bots and AI agents to research integrity in academia.
#AcademicSky #Research
See you in Chicago!
If you're heading to the @spspnews.bsky.social Annual Convention from Feb 26-28th, stop by booth #19 to find out how you can use Prolific’s new bot detection tool in your next study.
🎁 $50 credits offer available for first-timers. #AcademicSky #SocialPsyc #Research
📺 Watch part 1 now on YouTube → youtu.be/5vyiZXuDcSo
🎧 Listen on Spotify → open.spotify.com/episode/2Kmw...
On the first episode of The Frontier, we spoke to Crystal Qian, Senior Research Scientist at Google DeepMind about Deliberate Lab: an open-source platform for behavioral human-AI social experiments. 🧪
Developed by the PAIR (People + AI Research) team.
#MLSky #AcademicSky #ArtificialIntelligence
AI agents are becoming a serious threat to research data quality.
Today we’re rolling out Bot authenticity checks on @joinprolific.bsky.social, detecting agentic AI with 100% accuracy in testing.
Comes with a native Qualtrics integration! More info:
www.prolific.com/resources/in...
As new threats to data integrity emerge, we’ll continuously evolve our systems to tackle them.
Read more and share your feedback: www.prolific.com/resources/in... 🔗
Introducing bot authenticity checks that detect agentic AI and bots in academic research.
100% accuracy in testing. Our tool flags automated browser scripts or non-human behavior within your Qualtrics studies, so you can collect data with high confidence that it's human.
#AcademicSky #Research
What drives retention in online longitudinal research? We conducted an experiment (N=1,798) on @joinprolific.bsky.social, orthogonally manipulating payment rate (£6–£9/hr + bonus) and session interval (1, 2, or 4 weeks) across five sessions. The findings challenge some common assumptions 👇
When platforms consolidate, research options shrink.
Researchers are rightly asking whether participant access will stay tool-agnostic, high-quality, and reliable.
As @phe-lim.bsky.social says, we believe access works best when it’s open by design. So if you're reassessing, get in touch. #Research
AI modelling, product research, and academic studies all hinge on one thing most platforms can’t guarantee: real, trustworthy human data. 👥
This is where @joinprolific.bsky.social comes in, the platform-model tech company filling a critical gap in AI and research by providing fast,... (1/11)
Thank you 2025. You’ve been great. ⭐️
Together, our global community across AI, research, and participants made this a truly monumental year.
Whether you were collecting high-quality data or contributing your expertise, we were honored to be your human data platform of choice.
Thanks! We've flagged this to our support team
Here's every current method we use to ensure you receive authentic human data 👉 www.prolific.com/resources/ho...
As always, we invite feedback and new suggestions to help us stay ahead.
This multi-layered approach includes:
- Verifying new participants during onboarding,
- In-app participant monitoring
- Helping researchers verify participants inside studies
- Continuous development of our systems
Our platform is built on robust defensive measures, from rigorous onboarding and ID checks to high-precision authenticity detection in studies.
These focus on catching bots, agentic AI, and participants who use AI tools without consent.
AI agents are becoming more sophisticated, more capable of mimicking human behavior, more accessible.
This raises questions about how we keep real human feedback at the core of online research. 🧵 #AcademicSky #ResearchIntegrity
