A universal method for evaluating the quality of aggregators
Journal Title: Judgment and Decision Making - Year 2019, Vol 14, Issue 4
Abstract
We propose a new method to facilitate comparison of aggregated forecasts based on different aggregation, elicitation and calibration methods. Aggregates are evaluated by their relative position on the cumulative distribution of the corresponding individual scores. This allows one to compare methods using different measures of quality that use different scales. We illustrate the use of the method by re-analyzing various estimates from Budescu and Du (Management Science, 2007).
Authors and Affiliations
Ying Han and David Budescu
Keywords
Related Articles
- EP ID EP678426
- DOI -
- Views 178
- Downloads 0
The effect of incentive structure on search in the secretary problem
We tested the effectiveness of performance-based incentive structures using three incentive structures — commission base, best only and flat fee — and two levels of context — no context and house-selling — in an experime...
Is saving lives your task or God’s? Religiosity, belief in god, and moral judgment
Should a Catholic hospital abort a life-threatening pregnancy or let a pregnant woman die? Should a religious employer allow his employees access to contraceptives or break with healthcare legislation? People and organiz...
Context effects in games: Local versus global sequential effects on choice in the prisoner's dilemma game
We report an experiment exploring sequential context effects on strategy choices in one-shot Prisoner's Dilemma (PD) game. Rapoport and Chammah (1965) have shown that some PDs are cooperative and lead to high cooperation...
Performance on the Cognitive Reflection Test is stable across time
A widely used measure of individual propensity to utilize analytic processing is the Cognitive Reflection Test (CRT), a set of math problems with intuitively compelling but incorrect answers. Here, we ask whether scores...
Learning to reason: The influence of instruction, prompts and scaffolding, metacognitive knowledge, and general intelligence on informal reasoning about everyday social and political issues
Twelve experiments examined ways of improving informal reasoning, as assesed by presenting students with accessible, current, and interesting social and political issues, eliciting reasoning about them, and scoring the r...