The relationship between crowd majority and accuracy for binary decisions
Journal Title: Judgment and Decision Making - Year 2017, Vol 12, Issue 4
Abstract
We consider the wisdom of the crowd situation in which individuals make binary decisions, and the majority answer is used as the group decision. Using data sets from nine different domains, we examine the relationship between the size of the majority and the accuracy of the crowd decisions. We find empirically that these calibration curves take many different forms for different domains, and the distribution of majority sizes over decisions in a domain also varies widely. We develop a growth model for inferring and interpreting the calibration curve in a domain, and apply it to the same nine data sets using Bayesian methods. The modeling approach is able to infer important qualitative properties of a domain, such as whether it involves decisions that have ground truths or are inherently uncertain. It is also able to make inferences about important quantitative properties of a domain, such as how quickly the crowd accuracy increases as the size of the majority increases. We discuss potential applications of the measurement model, and the need to develop a psychological account of the variety of calibration curves that evidently exist.
Authors and Affiliations
Michael D. Lee and Megan N. Lee
Keywords
Related Articles
- EP ID EP678298
- DOI -
- Views 173
- Downloads 0
Anticipatory stress interferes with utilitarian moral judgment
A recent study indicates that acute stress affects moral decision making (Youssef et al., in press). The current study examines whether results can be replicated using a different kind of stressor and a different kind of...
TEMAP2.R: True and Error model analysis program in R
True and Error Theory (TET) provides a method to separate the variability of behavior into components due to changing true policy and to random error. TET is a testable theory that can serve as a statistical model, allow...
Framing effects on bidding behavior in experimental first-price sealed-bid money auctions
Consumers often face prices that are the sum of two components, for example, an online purchase that includes a stated price and shipping costs. In such cases consumer behavior may be influenced by framing, i.e., how the...
Aggregating multiple probability intervals to improve calibration
We apply the principles of the “Wisdom of Crowds (WoC)” to improve the calibration of interval estimates. Previous research has documented the significant impact of the WoC on the accuracy of point estimates but only a f...
Prosociality in the economic Dictator Game is associated with less parochialism and greater willingness to vote for intergroup compromise
Is prosociality parochial or universalist? To shed light on this issue, we examine the relationship between the amount of money given to a stranger (giving in an incentivized Dictator Game) and intergroup attitudes and b...