Prolific

For future we're continuing to invest and develop. Highly recommend this article where we share more about our current and evolving approaches: www.prolific.com/resources/pr...

Any feedback you have on that is appreciated.

Thanks @jessewise.bsky.social for raising. Our upgraded system addresses AI-generated responses. We run 50+ onboarding checks (including bank-grade ID verification), always-on monitoring such as periodic video identity checks and regular demographic refreshes, and offer in-study detection tools.

Lots of chatter about this paper currently. Its a stark warning, but at present I see this as a stark warning of what might come, not what is happening now. As a research community we need to see it as a call-to-arms to develop new strategies, NOT a call to abandon online sampling. Reasoning below

Without minimising the seriousness of the threat raised in this paper, I'm more optimistic. This is just the latest challenge in online integrity of online research.

We've been proactively adding to our suite of authenticity tools - more every week - including many of Sean's recommendations:

As part of our commitment to maintaining integrity, we believe ongoing collaboration is fundamental. If you share our vision for clean human data, reach out.

Full report here: www.insightsassociation.org/Education/Ed...

#AcademicSky #ResearchIntegrity

We’re proud to surpass multiple industry benchmarks in the Insights Association’s recent Global Data Quality (GDQ) Benchmarking Report for online #research data quality.

Our metrics include:

⭐️ 2.7% pre-survey removal rate
⭐️ 0.5% in-survey behavior removal
⭐️ 0.5% post-survey cleanout rate

There's not just a generational wealth gap anymore. There's a wellbeing gap too.

And right now, it's hitting younger people the hardest.

(Data from our latest @joinprolific.bsky.social poll - November 2024)

Full data: www.prolific.com/polling

Find out more about our approach with full details of each of the uncompromising layers of protection: www.prolific.com/resources/pr...

Protocol is our five-layer system built to ensure your data is of the highest quality.

VP of Product Sara Saab guides you through the rigorous identity and quality checks, in-study detection tools, and more that ensure your research is powered by human intelligence, always.

#AcademicSky #Research

Don’t wait to discover your data is contaminated. You should work with a data partner that refuses to compromise on what matters most: data integrity.

Learn more: www.prolific.com/resources/pr... 📰

Ongoing human oversight and fine-tuning of our data quality systems is also crucial in protecting good participants and research integrity.

Our system benefits everyone. You get reliable data, and our valued participants enjoy a platform free from unfair competition.

We architect authentic data through Protocol, a state-of-the-art system that sets the industry standard for data quality. With five layers of defense, it ensures every response on our platform is authentically human.

Each layer is detailed in the full article.

We put authenticity above all else.

Independent research validates Prolific as setting the standard for data quality (2017-2025). Only 0.5% of Prolific participant responses were rejected in 2025. And a recent audit shows that just 0.99% of participants were flagged for AI-generated responses.

When you commission human data, you deserve human data.

Our commitment to data integrity remains constant. Protecting access to real human perspectives has been Prolific’s top priority since its inception in 2014.

In an AI-powered world, genuine human perspectives are both precious and vulnerable.

The line between human and AI-generated responses is blurring, threatening the very foundation of academic research.

Our approach to protecting data integrity 🧵 #AcademicSky #Research

🎓 Great news for academic researchers - our $100 / £100 free credit offer has been extended!

Sign up with your university email to collect fast, high-quality data from 200K+ vetted participants across the globe.

👉 www.prolific.com/academic-res...

#AcademicSky #PhDSky #AcademicResearch

What is one question you'd dearly love to ask a representative sample of 2,000 US adults?

Going to try this again because it was super interesting last time. I'll stick the best question in our next @joinprolific.bsky.social poll that goes live at the start of November.

#polling #research

Limited time offer for academic researchers 🚨

Get $100 / £100 free credit this October to run your next study with fast, high-quality data from 200K+ verified participants on Prolific.

Sign up and redeem: www.prolific.com/academic-res...

#AcademicSky #PhDSky #Researchers

The Double Squeeze hitting Trump's base.

9 months of @joinprolific.bsky.social tracking show voters report surging costs AND growing fears of US global decline.

They voted for prosperity & strength. They got strain & decline.

Full analysis: www.prolific.com/resources/tr...

#poll #geopolitics

How to design studies and get quality data with Dr. Simon Jones This webinar explores how, as AI transforms how we interact with tech, the human role is shifting from data provider to strategic designer.

How to design studies and get quality data with Dr. Simon Jones 🎥

Join our Senior Research Consultant's refresher as he shares expert tips to optimize data quality, and runs through the latest platform features designed to make launching studies faster and easier than ever.

#AcademicSky #Research

Note that the tool doesn't automatically reject anyone's submission - researchers are advised to check responses and make an informed decision to approve or reject. If rejected, this doesn't result in a ban either unless the participant has a high number of rejections overall. Hope this helps.

Hi team, the false positive rate is 0.6%. We take every measure to ensure a very low false positive rate and provide extensive guidance, but if a participant feels they were incorrectly flagged or rejected, we advise they make their case to the researcher first, or contact support if unresolved.

In the age of AI, we’re committed to preserving genuine human insights that power impactful academic research.

Try it and give us your feedback: www.prolific.com/resources/in... 🔗

📢 Authenticity checks are out of beta, with upgrades for ease of use.

Authenticity checks work with Qualtrics and Prolific to detect AI-generated responses in your research with 98.7% precision, so you can trust the data you collect is authentic.

#AcademicSky #ResearchIntegrity #AIDetection

27 AI models were ranked by the public and ChatGPT came 8th — these are the models that beat it A surprising set of results for the big AI chatbots

AI benchmarks show models like GPT-5 might ace scientific reasoning, or Gemini and Claude adapt to new concepts. But which LLMs offer the best user experience?

@tomsguide.com explains how Prolific's AI leaderboard HUMAINE is a big step forward.

#ArtificialIntelligence #MachineLearning #LLM

Super excited to share this work with the world!

We wanted to put human preference at the heart of our AI model leaderboard, and to do so in a principled and representative manner.

The result: HUMAINE

Check out the data here and get in touch with any questions: huggingface.co/spaces/Proli...

HUMAINE Leaderboard - a Hugging Face Space by ProlificAI This application helps you analyze human feedback to evaluate AI models. You provide feedback data, and it gives you insights to improve your models.

Get more insights, compare models, and give us a like on Hugging Face ➡️ huggingface.co/spaces/Proli...

Follow for updates on requested models.

Probability of Practical Superiority (PPS) Comparison Matrix is available under "Model Comparison" for estimating how likely one model is likely to outperform another in real-world usage.

Values >55% suggest one model meaningfully outperforms beyond random chance.

HUMAINE uses rigorous methodology to ensure representative and reliable results.

Our methodology includes:

- Comparative assessment
- Statistically rigorous hierarchical modeling
- Four core dimensions of user experience and overall winner
- Commitment to demographic representation

🏆 Gemini-2.5-Pro takes the #1 spot decisively, with DeepSeek-V3 emerging as the surprise runner-up.

⚖️ Age groups show highest disagreement on model rankings.

📊 27 models tested across 100K+ human comparisons - one of the largest public AI evaluations to date.