Aggregating multiple probability intervals to improve calibration
Journal Title: Judgment and Decision Making - Year 2015, Vol 10, Issue 2
Abstract
We apply the principles of the “Wisdom of Crowds (WoC)” to improve the calibration of interval estimates. Previous research has documented the significant impact of the WoC on the accuracy of point estimates but only a few studies have examined its effectiveness in aggregating interval estimates. We demonstrate that collective probability intervals obtained by several heuristics can reduce the typical overconfidence of the individual estimates. We re-analyzed data from Glaser, Langer and Weber (2013) and from Soll and Klayman (2004) and applied four heuristics Averaging, Median, Enveloping, Probability averaging-suggested by Gaba, Tsetlin and Winkler (2014) and new heuristics, Averaging with trimming and Quartiles. We used the hit rate and the Mean Squared Error (MSE) to evaluate the quality of the methods. All methods reduced miscalibration to some degree, and Quartiles was the most beneficial securing accuracy and informativeness.
Authors and Affiliations
Saemi Park and David V. Budescu
Keywords
Related Articles
- EP ID EP678166
- DOI -
- Views 119
- Downloads 0
New paradoxes in intertemporal choice
Similar to research on risky choice, the traditional analysis of intertemporal choice takes the view that an individual behaves so as to maximize the discounted sum of all future utilities. The well-known Allais paradox...
Contamination without contact: An examination of intention-based contagion
Contagion refers to the belief that individuals or objects can acquire the essence of a particular source, such as a disgusting product or an immoral person, through physical contact. This paper documents beliefs in a "c...
The recognition heuristic: A decade of research
The recognition heuristic exploits the basic psychological capacity for recognition in order to make inferences about unknown quantities in the world. In this article, we review and clarify issues that emerged from our i...
Why dyads heed advice less than individuals do
Following up on a recent debate, we examined advice taking in dyads compared to individuals in a set of three studies (total N = 303 dyads and 194 individuals). Our first aim was to test the replicability of an important...
Is loss-aversion magnitude-dependent? Measuring prospective affective judgments regarding gains and losses
Prospect Theory proposed that the (dis)utility of losses is always more than gains due to a phenomena called ‘loss-aversion’, a result obtained in multiple later studies over the years. However, some researchers found re...