How generalizable is good judgment? A multi-task, multi-benchmark study
Journal Title: Judgment and Decision Making - Year 2017, Vol 12, Issue 4
Abstract
Good judgment is often gauged against two gold standards – coherence and correspondence. Judgments are coherent if they demonstrate consistency with the axioms of probability theory or propositional logic. Judgments are correspondent if they agree with ground truth. When gold standards are unavailable, silver standards such as consistency and discrimination can be used to evaluate judgment quality. Individuals are consistent if they assign similar judgments to comparable stimuli, and they discriminate if they assign different judgments to dissimilar stimuli. We ask whether “superforecasters”, individuals with noteworthy correspondence skills (see Mellers et al., 2014) show superior performance on laboratory tasks assessing other standards of good judgment. Results showed that superforecasters either tied or out-performed less correspondent forecasters and undergraduates with no forecasting experience on tests of consistency, discrimination, and coherence. While multifaceted, good judgment may be a more unified than concept than previously thought.
Authors and Affiliations
Barbara A. Mellers, Joshua D. Baker, Eva Chen, David R. Mandel and Philip E. Tetlock
Keywords
Related Articles
- EP ID EP678300
- DOI -
- Views 159
- Downloads 0
Rebate subsidies, matching subsidies and isolation effects
In a series of recent experiments (Davis, Millner and Reilly, 2005, Eckel and Grossman, 2003, 2005a-c, 2006), matching subsidies generate significantly higher charity receipts than do theoretically equivalent rebate sub...
Testing the effect of time pressure on asymmetric dominance and compromise decoys in choice
Dynamic, connectionist models of decision making, such as decision field theory (Roe, Busemeyer, & Townsend, 2001), propose that the effect of context on choice arises from a series of pairwise comparisons between attrib...
Affect, risk perception and future optimism after the tsunami disaster
Environmental events such as natural disasters may influence the public's affective reactions and decisions. Shortly after the 2004 Tsunami disaster we assessed how affect elicited by thinking about this disaster influen...
Is saving lives your task or God’s? Religiosity, belief in god, and moral judgment
Should a Catholic hospital abort a life-threatening pregnancy or let a pregnant woman die? Should a religious employer allow his employees access to contraceptives or break with healthcare legislation? People and organiz...
Comparative evaluation of the forecast accuracy of analysis reports and a prediction market
This paper summarizes an empirical comparison of the accuracy of forecasts included in analysis reports developed by professional intelligence analysts to comparable forecasts in a prediction market that has broad partic...