Mission Possible: The Collection of High-Quality Data

Absent high-quality online data, research questions would be constrained conceptually and in study populations. To inform the debate about online data quality, this paper provides empirical evidence that compares data quality of responses from online participants, AI agents, and human subjects in the lab. Corresponding results reveal high data quality on some platforms, but not others. This paper also highlights a viable path for high-quality online data in an evolving landscape: use a two-stage recruitment method to broadly recruit online subjects in a baseline study and then limit recruitment for the main study to the resulting subset of "high quality" subjects.