A model-based approach for the analysis of the calibration of probability judgments
Journal Title: Judgment and Decision Making - Year 2011, Vol 6, Issue 8
Abstract
The calibration of probability or confidence judgments concerns the association between the judgments and some estimate of the correct probabilities of events. Researchers rely on estimates using relative frequencies computed by aggregating data over observations. We show that this approach creates conceptual problems, and may result in the confounding of explanatory variables or unstable estimates. To circumvent these problems we propose using probability estimates obtained from statistical models—specifically mixed models for binary data—in the analysis of calibration. We illustrate this methodology by re-analyzing data from a published study and comparing the results from this approach to those based on relative frequencies. The model-based estimates avoid problems with confounding variables and provided more precise estimates, resulting in better inferences.
Authors and Affiliations
David V. Budescu and Timothy R. Johnson
Keywords
Related Articles
- EP ID EP677868
- DOI -
- Views 131
- Downloads 0
A marketing science perspective on recognition-based heuristics (and the fast-and-frugal paradigm)
Marketing science seeks to prescribe better marketing strategies (advertising, product development, pricing, etc.). To do so we rely on models of consumer decisions grounded in empirical observations. Field experience su...
The collective intelligence of random small crowds: A partial replication of Kosinski et al. (2012)
We examined the trade-off between the cost of response redundancy and the gain in output quality on the popular crowdsourcing platform Mechanical Turk, as a partial replication of Kosinski et al. (2012) who demonstrated...
Cross-national in-group favoritism in prosocial behavior: Evidence from Latin and North America
As individuals from different nations increasingly interact with each other, research on national in-group favoritism becomes particularly vital. In a cross-national, large-scale study (N = 915) including representative...
Cognitive reflection predicts the acceptance of unfair ultimatum game offers
In the ultimatum game, one player proposes a split of money between him- or herself and another player, who can accept the offer (and both players keep the allocated money) or reject the offer (and both players get nothi...
Searching for coherence in a correspondence world
In this paper, I trace the evolution of the aircraft cockpit as an example of the transformation of a probabilistic environment into an ecological hybrid, that is, an environment characterized by both probabilistic and d...