The relationship between crowd majority and accuracy for binary decisions
Journal Title: Judgment and Decision Making - Year 2017, Vol 12, Issue 4
Abstract
We consider the wisdom of the crowd situation in which individuals make binary decisions, and the majority answer is used as the group decision. Using data sets from nine different domains, we examine the relationship between the size of the majority and the accuracy of the crowd decisions. We find empirically that these calibration curves take many different forms for different domains, and the distribution of majority sizes over decisions in a domain also varies widely. We develop a growth model for inferring and interpreting the calibration curve in a domain, and apply it to the same nine data sets using Bayesian methods. The modeling approach is able to infer important qualitative properties of a domain, such as whether it involves decisions that have ground truths or are inherently uncertain. It is also able to make inferences about important quantitative properties of a domain, such as how quickly the crowd accuracy increases as the size of the majority increases. We discuss potential applications of the measurement model, and the need to develop a psychological account of the variety of calibration curves that evidently exist.
Authors and Affiliations
Michael D. Lee and Megan N. Lee
Keywords
Related Articles
- EP ID EP678298
- DOI -
- Views 160
- Downloads 0
New findings on unconscious versus conscious thought in decision making: additional empirical data and meta-analysis.
Ninety-eight Australian students participated in a functional replication of a study published by Dijksterhuis et al. (2006). The results indicated that unconscious thought does not necessarily lead to better normative d...
Memory retrieval processes help explain the incumbency advantage
Voters prefer political candidates who are currently in office (incumbents) over new candidates (challengers). Using the premise of query theory (Johnson, Häubl & Keinan, 2007), we clarify the underlying cognitive mechan...
Communicating clinical trial outcomes: Effects of presentation method on physicians’ evaluations of new treatments
Physicians expect a treatment to be more effective when its clinical outcomes are described as relative rather than as absolute risk reductions. We examined whether effects of presentation method (relative vs. absolute r...
Cognitive reflection test and behavioral biases in Malaysia
We asked whether behavioral biases are related to cognitive abilities of Malaysian youth. Frederick’s three-item Cognitive Reflection Test was used to understand the role of behavioral biases concerning behavioral econom...
Deception and price in a market with asymmetric information
In markets with asymmetric information, only sellers have knowledge about the quality of goods. Sellers may of course make a declaration of the quality, but unless there are sanctions imposed on false declarations or rep...