A universal method for evaluating the quality of aggregators
Journal Title: Judgment and Decision Making - Year 2019, Vol 14, Issue 4
Abstract
We propose a new method to facilitate comparison of aggregated forecasts based on different aggregation, elicitation and calibration methods. Aggregates are evaluated by their relative position on the cumulative distribution of the corresponding individual scores. This allows one to compare methods using different measures of quality that use different scales. We illustrate the use of the method by re-analyzing various estimates from Budescu and Du (Management Science, 2007).
Authors and Affiliations
Ying Han and David Budescu
Keywords
Related Articles
- EP ID EP678426
- DOI -
- Views 173
- Downloads 0
Self-reported ethical risk taking tendencies predict actual dishonesty
Are people honest about the extent to which they engage in unethical behaviors? We report an experiment examining the relation between self-reported risky unethical tendencies and actual dishonest behavior. Participants’...
Do we de-bias ourselves?: The impact of repeated presentation on the bat-and-ball problem
The notorious bat-and-ball problem has long been used to demonstrate that people are easily biased by their intuitions. In this paper we test the robustness of biased responding by examining how it is affected by repeate...
A simple remedy for overprecision in judgment
Overprecision is the most robust type of overconfidence. We present a new method that significantly reduces this bias and offers insight into its underlying cause. In three experiments, overprecision was significantly re...
The effects of behavioral and outcome feedback on prudent decision-making under conditions of present and future uncertainty
One of the largest reasons decision-makers make bad decisions (act imprudently) is that the world is full of uncertainty, we feel uncertain about the consequences of our actions. Participants played a repeated game in wh...
Hold on to it? An experimental analysis of the disposition effect
This paper experimentally investigates a well-known anomaly in portfolio management, i.e., the fact that paper losses are realized less than paper gains (disposition effect). I confirm the existence of the disposition ef...