Using inferred probabilities to measure the accuracy of imprecise forecasts
Journal Title: Judgment and Decision Making - Year 2012, Vol 7, Issue 6
Abstract
Research on forecasting is effectively limited to forecasts that are expressed with clarity; which is to say that the forecasted event must be sufficiently well-defined so that it can be clearly resolved whether or not the event occurred and forecasts certainties are expressed as quantitative probabilities. When forecasts are expressed with clarity, then quantitative measures (scoring rules, calibration, discrimination, etc.) can be used to measure forecast accuracy, which in turn can be used to measure the comparative accuracy of different forecasting methods. Unfortunately most real world forecasts are not expressed clearly. This lack of clarity extends to both the description of the forecast event and to the use of vague language to express forecast certainty. It is thus difficult to assess the accuracy of most real world forecasts, and consequently the accuracy the methods used to generate real world forecasts. This paper addresses this deficiency by presenting an approach to measuring the accuracy of imprecise real world forecasts using the same quantitative metrics routinely used to measure the accuracy of well-defined forecasts. To demonstrate applicability, the inferred probability method is applied to measure the accuracy of forecasts in fourteen documents examining complex political domains.
Authors and Affiliations
Paul Lehner, Avra Michelson, Leonard Adelman and Anna Goodman
Keywords
Related Articles
- EP ID EP678029
- DOI -
- Views 158
- Downloads 0
Hedonic “adaptation”: Specific habituation to disgust/death elicitors as a result of dissecting a cadaver
People live in a world in which they are surrounded by potential disgust elicitors such as “used” chairs, air, silverware, and money as well as excretory activities. People function in this world by ignoring most of thes...
The use of mixed models in a modified Iowa Gambling Task and a prisoner's dilemma game
Researchers in the decision making tradition usually analyze multiple decisions within experiments by aggregating choices across individuals and using the individual subject as the unit of analysis. This approach can mas...
American’s desire for less wealth inequality does not depend on how you ask them
A large body of survey research offers evidence that citizens are not always fully aware of the economic and political realities in their respective countries. Norton and Ariely (2011) extended this research to the domai...
Who makes utilitarian judgments? The influences of emotions on utilitarian judgments
Recent research has emphasized emotion’s role in non-utilitarian judgments, but has not focused much on characteristics of subjects contributing to those judgments. The present article relates utilitarian judgment to ind...
Religion, group threat and sacred values
Sacred or protected values have important influences on decision making, particularly in the context of intergroup disputes. Thus far, we know little about the process of a value becoming sacred or why one person may be...