How generalizable is good judgment? A multi-task, multi-benchmark study
Journal Title: Judgment and Decision Making - Year 2017, Vol 12, Issue 4
Abstract
Good judgment is often gauged against two gold standards – coherence and correspondence. Judgments are coherent if they demonstrate consistency with the axioms of probability theory or propositional logic. Judgments are correspondent if they agree with ground truth. When gold standards are unavailable, silver standards such as consistency and discrimination can be used to evaluate judgment quality. Individuals are consistent if they assign similar judgments to comparable stimuli, and they discriminate if they assign different judgments to dissimilar stimuli. We ask whether “superforecasters”, individuals with noteworthy correspondence skills (see Mellers et al., 2014) show superior performance on laboratory tasks assessing other standards of good judgment. Results showed that superforecasters either tied or out-performed less correspondent forecasters and undergraduates with no forecasting experience on tests of consistency, discrimination, and coherence. While multifaceted, good judgment may be a more unified than concept than previously thought.
Authors and Affiliations
Barbara A. Mellers, Joshua D. Baker, Eva Chen, David R. Mandel and Philip E. Tetlock
Keywords
Related Articles
- EP ID EP678300
- DOI -
- Views 140
- Downloads 0
Decision-making styles and depressive symptomatology: Development of the Decision Styles Questionnaire
Difficulty making decisions is one of the symptoms of the depressive illness. Previous research suggests that depressed individuals may make decisions that differ from those made by the non-depressed, and that they use s...
A new intuitionism: Meaning, memory, and development in Fuzzy-Trace Theory
Combining meaning, memory, and development, the perennially popular topic of intuition can be approached in a new way. Fuzzy-trace theory integrates these topics by distinguishing between meaning-based gist representatio...
Applying the decision moving window to risky choice: Comparison of eye-tracking and mouse-tracing methods
Currently, a disparity exists between the process-level models decision researchers use to describe and predict decision behavior and the methods implemented and metrics collected to test these models. The current work s...
A note on determining the number of cues used in judgment analysis studies: The issue of type II error
Many judgment analysis studies employ multiple regression procedures to estimate the importance of cues. Some studies test the significance of regression coefficients in order to decide whether or not specific cues are a...
Wording effects in moral judgments
As the study of moral judgments grows, it becomes imperative to compare results across studies in order to create unified theories within the field. These efforts are potentially undermined, however, by variations in wor...