Using cognitive models to combine probability estimates
Journal Title: Judgment and Decision Making - Year 2014, Vol 9, Issue 3
Abstract
We demonstrate the usefulness of cognitive models for combining human estimates of probabilities in two experiments. The first experiment involves people’s estimates of probabilities for general knowledge questions such as “What percentage of the world’s population speaks English as a first language?” The second experiment involves people’s estimates of probabilities in football (soccer) games, such as “What is the probability a team leading 1–0 at half time will win the game?”, with ground truths based on analysis of large corpus of games played in the past decade. In both experiments, we collect people’s probability estimates, and develop a cognitive model of the estimation process, including assumptions about the calibration of probabilities and individual differences. We show that the cognitive model approach outperforms standard statistical aggregation methods like the mean and the median for both experiments and, unlike most previous related work, is able to make good predictions in a fully unsupervised setting. We also show that the parameters inferred as part of the cognitive modeling, involving calibration and expertise, provide useful measures of the cognitive characteristics of individuals. We argue that the cognitive approach has the advantage of aggregating over latent human knowledge rather than observed estimates, and emphasize that it can be applied in predictive settings where answers are not yet available.
Authors and Affiliations
Michael D. Lee and Irina Danileiko
Keywords
Related Articles
- EP ID EP678127
- DOI -
- Views 149
- Downloads 0
Process dynamics in delay discounting decisions: An attractor dynamics approach
How do people make decisions between an immediate but small reward and a delayed but large one? The outcome of such decisions indicates that people discount rewards by their delay and hence these outcomes are well descri...
This way, please: Uncovering the directional effects of attribute translations on decision making
The translation of choice attributes into more meaningful information (e.g., from kWh to costs) is a form of choice architecture that is thought to facilitate decision making by providing decision signposts that activate...
Action orientation, consistency and feelings of regret
Previous research has demonstrated that consistency between people's behavior and their dispositions has predictive validity for judgments of regret. Research has also shown that differences in the personality variable o...
Actor/observer asymmetry in risky decision making
Are people willing to gamble more for themselves than what they deem reasonable for others? We addressed this question in a simplified computer gambling task in which subjects chose from a set of 10 cards. Subjects selec...
Moral judgments of risky choices: A moral echoing effect
Two experiments examined moral judgments about a decision-maker’s choices when he chose a sure-thing, 400 out of 600 people will be saved, or a risk, a two-thirds probability to save everyone and a one-thirds probability...