Brian and Andrew sit down with Karine Pepin of The Research Heads and Andrew Gordon of Prolific to break down one of the most important data quality studies the industry has produced. They analyzed 5,200 respondents across 10 platforms and the findings will surprise you.

AI agents are not in your data. Outside of MTurk, detection rates were below 0.5% across every platform type. And the things getting flagged on MTurk? Old-school bots, not LLMs. When they ran actual AI agents through the same survey, those agents outperformed real human respondents on nearly every quality measure.

The actual crisis is your human respondents and the platform you are buying them from.

Key Takeaways:

• Agent infiltration is not the crisis the industry thinks it is

• What is actually getting flagged as bots on MTurk, and what it really means

• Why the human quality gap between direct panels & marketplace platforms is the story nobody is telling

• Why the cheapest sample per click is often the most expensive data you will ever buy

Link to the study "AI Agent Prevalence and Data Quality Across Multiple Online Sample Providers"

If you loved the episode, have comments, or want to appear on the show, connect with us down below!

Connect with us:

• LinkedIn

• YouTube

• ROI Rocket

Connect with Karine Pepin:

• LinkedIn

Connect with Andrew Gordon:

• LinkedIn