Running experiments on Amazon Mechanical Turk
Journal Title: Judgment and Decision Making - Year 2010, Vol 5, Issue 5
Abstract
Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online and offline methods of recruiting subjects, and comparing the magnitude of effects obtained using Mechanical Turk and traditional subject pools. We further discuss some additional benefits such as the possibility of longitudinal, cross cultural and prescreening designs, and offer some advice on how to best manage a common subject pool.
Authors and Affiliations
Gabriele Paolacci, Jesse Chandler and Panagiotis G. Ipeirotis
Keywords
Related Articles
- EP ID EP677756
- DOI -
- Views 113
- Downloads 0
Magical thinking in predictions of negative events: Evidence for tempting fate but not for a protection effect
In this paper we test two hypotheses regarding magical thinking about the perceived likelihood of future events. The first is that people believe that those who “tempt fate” by failing to take necessary precautions are m...
The role of cognitive reflection in decision making: Evidence from Pakistani managers
Assessing how managers discount and evaluate risks is crucial in designing effective managerial policies. In this work, we examine whether risk preferences (RP; both in the domains of gain and loss) and time preferences...
The curious tale of Julie and Mark: Unraveling the moral dumbfounding effect
The paper critically reexamines the well-known “Julie and Mark” vignette, a stylized account of two college-age siblings opting to engage in protected sex while vacationing abroad (e.g., Haidt, 2001). Since its inception...
A medical risk attitude subscale for DOSPERT
Background: The Domain-Specific Risk Taking scale (DOSPERT) is a widely used instrument that measures perceived risk and benefit and attitude toward risk for activities in several domains, but does not include medical ri...
The psychology of moral reasoning
This article presents a theory of reasoning about moral propositions that is based on four fundamental principles. First, no simple criterion picks out propositions about morality from within the larger set of deontic pr...