Aggregating multiple probability intervals to improve calibration
Journal Title: Judgment and Decision Making - Year 2015, Vol 10, Issue 2
Abstract
We apply the principles of the “Wisdom of Crowds (WoC)” to improve the calibration of interval estimates. Previous research has documented the significant impact of the WoC on the accuracy of point estimates but only a few studies have examined its effectiveness in aggregating interval estimates. We demonstrate that collective probability intervals obtained by several heuristics can reduce the typical overconfidence of the individual estimates. We re-analyzed data from Glaser, Langer and Weber (2013) and from Soll and Klayman (2004) and applied four heuristics Averaging, Median, Enveloping, Probability averaging-suggested by Gaba, Tsetlin and Winkler (2014) and new heuristics, Averaging with trimming and Quartiles. We used the hit rate and the Mean Squared Error (MSE) to evaluate the quality of the methods. All methods reduced miscalibration to some degree, and Quartiles was the most beneficial securing accuracy and informativeness.
Authors and Affiliations
Saemi Park and David V. Budescu
Keywords
Related Articles
- EP ID EP678166
- DOI -
- Views 168
- Downloads 0
Recognising the recognition heuristic for what it is (and what it’s not)
The diversity, ingenuity and differences of opinion displayed in the articles of the recent special issues on the recognition heuristic are testament to the power and theoretical fertility of a simple idea about the role...
How to study cognitive decision algorithms: The case of the priority heuristic
Although the priority heuristic (PH) is conceived as a cognitive-process model, some of its critical process assumptions remain to be tested. The PH makes very strong ordinal and quantitative assumptions about the strict...
Information search in everyday decisions: The generalizability of the attraction search effect
The recently proposed integrated coherence-based decisions and search model (iCodes) makes predictions for search behavior in multi-attribute decision tasks beyond those of classic decision-making heuristics. More precis...
The Risk-as-feelings hypothesis in a Theory-of-planned-behaviour perspective
The Theory of Planned Behaviour (TpB: Ajzen, 1985; 1991) is based on a utility framework, and the Risk-as-Feelings hypothesis (RaF: Loewenstein, Weber, Hsee, & Welch, 2001) is a feelings-based behavioural model. The TpB...
Response mode, compatibility, and dual-processes in the evaluation of simple gambles: An eye-tracking investigation
We employed simple gambles to investigate information processing in relation to the compatibility effect. Subjects should be more likely to engage in a deliberative thinking strategy when completing a pricing task rather...