Skip to main content

Mechanical Turk, Tim Brady

Dr. Tim Brady 
Psychology Department 
University of California, San Diego 
More about Dr. Brady's research 

Running experiments on Mechanical Turk

Mechanical Turk is a service provided Amazon that enables anybody to post short tasks and have participants from around the United States or the world complete them in exchange for payment. Workers can browse a huge set of tasks and choose which to complete. Only a small proportion of tasks on Turk are cognitive science experiments: the vast majority are short, simple tasks for business purposes (e.g., translating or transcribing speech; screening for inappropriate images, etc). 

In this talk, I'll give a brief overview of using Mechanical Turk as a large-scale subject pool for research. I'll review the pros and cons of Turk as a participant sample and as a platform for experiments. I'll review what we know about the demographics of Turk users and what we know about the validity of Turk for psychology experiments. I'll then explore the technical and practical limitations of running online experiments, and give a variety of examples of Turk studies, both from my own work (e.g., Brady & Alvarez, 2011, Psych. Science; Brady & Tenenbaum, 2013, Psych. Review) and from others.