Abstract
The Internet has enabled recruitment of large samples with specific
characteristics. However, when researchers rely on participant self-report
to determine eligibility, data quality depends on participant honesty.
Across four studies on Amazon Mechanical Turk, we show that a substantial
number of participants misrepresent theoretically relevant characteristics
(e.g., demographics, product ownership) to meet eligibility criteria
explicit in the studies or inferred by exclusion from the study on a first
attempt or in previous experiences with similar studies. When recruiting
rare populations, a large proportion of responses can be deceptive. We
conclude with recommendations about how to ensure that ineligible
participants are excluded that are applicable to a wide variety of data
collection efforts that rely on self-report.