A universal method for evaluating the quality of aggregators
Journal Title: Judgment and Decision Making - Year 2019, Vol 14, Issue 4
Abstract
We propose a new method to facilitate comparison of aggregated forecasts based on different aggregation, elicitation and calibration methods. Aggregates are evaluated by their relative position on the cumulative distribution of the corresponding individual scores. This allows one to compare methods using different measures of quality that use different scales. We illustrate the use of the method by re-analyzing various estimates from Budescu and Du (Management Science, 2007).
Authors and Affiliations
Ying Han and David Budescu
Keywords
Related Articles
- EP ID EP678426
- DOI -
- Views 172
- Downloads 0
Reversing the causal arrow: Incidence and properties of negative backward magical contagion in Americans
Backward magical contagion describes instances in which individuals (sources) express discomfort or pleasure when something connected to them (medium; e.g., hair, a diary) falls into the possession of a negatively- or po...
Diagnostic Task Selection for Strategy Classification in Judgment and Decision Making: Theory, Validation, and Implementation in R
One major statistical and methodological challenge in Judgment and Decision Making research is the reliable identification of individual decision strategies by selection of diagnostic tasks, that is, tasks for which pred...
Success-slope effects on the illusion of control and on remembered success-frequency
The illusion of control refers to the inference of action-outcome contingency in situations where outcomes are in fact random. The strength of this illusion has been found to be affected by whether the frequency of succe...
Perceived time pressure and the Iowa Gambling Task
The purpose of the present study was to investigate the effect of perceived time pressure on a learning-based task called the Iowa Gambling Task (IGT). One hundred and sixty-three participants were randomly assigned to o...
Approximating rationality under incomplete information: Adaptive inferences for missing cue values based on cue-discrimination
In a highly uncertain world, individuals often have to make decisions in situations with incomplete information. We investigated in three experiments how partial cue information is treated in complex probabilistic infere...