Using cognitive models to combine probability estimates
Journal Title: Judgment and Decision Making - Year 2014, Vol 9, Issue 3
Abstract
We demonstrate the usefulness of cognitive models for combining human estimates of probabilities in two experiments. The first experiment involves people’s estimates of probabilities for general knowledge questions such as “What percentage of the world’s population speaks English as a first language?” The second experiment involves people’s estimates of probabilities in football (soccer) games, such as “What is the probability a team leading 1–0 at half time will win the game?”, with ground truths based on analysis of large corpus of games played in the past decade. In both experiments, we collect people’s probability estimates, and develop a cognitive model of the estimation process, including assumptions about the calibration of probabilities and individual differences. We show that the cognitive model approach outperforms standard statistical aggregation methods like the mean and the median for both experiments and, unlike most previous related work, is able to make good predictions in a fully unsupervised setting. We also show that the parameters inferred as part of the cognitive modeling, involving calibration and expertise, provide useful measures of the cognitive characteristics of individuals. We argue that the cognitive approach has the advantage of aggregating over latent human knowledge rather than observed estimates, and emphasize that it can be applied in predictive settings where answers are not yet available.
Authors and Affiliations
Michael D. Lee and Irina Danileiko
Keywords
Related Articles
- EP ID EP678127
- DOI -
- Views 168
- Downloads 0
Biased calculations: Numeric anchors influence answers to math equations
People must often perform calculations in order to produce a numeric estimate (e.g., a grocery-store shopper estimating the total price of his or her shopping cart contents). The current studies were designed to test whe...
Political double standards in reliance on moral foundations
Prior research using the Moral Foundations Questionnaire (MFQ) has established that political ideology is associated with self-reported reliance on specific moral foundations in moral judgments of acts. MFQ items do not...
Facing expectations: Those that we prefer to fulfil and those that we disregard.
We argue that people choosing prosocial distribution of goods (e.g., in dictator games) make this choice because they do not want to disappoint their partner rather than because of a direct preference for the chosen pros...
Deliberation versus automaticity in decision making: Which presentation format features facilitate automatic decision making?
The idea of automatic decision making approximating normatively optimal decisions without necessitating much cognitive effort is intriguing. Whereas recent findings support the notion that such fast, automatic processes...
On the meaning and measurement of maximization
Building on Herbert Simon’s critique of rational choice theory, Schwartz et al. (2002) proposed that when making choices, some individuals — maximizers — search extensively through many alternatives with the goal of maki...