Using inferred probabilities to measure the accuracy of imprecise forecasts

Apply

Using inferred probabilities to measure the accuracy of imprecise forecasts

Journal Title: Judgment and Decision Making - Year 2012, Vol 7, Issue 6

Abstract

Research on forecasting is effectively limited to forecasts that are expressed with clarity; which is to say that the forecasted event must be sufficiently well-defined so that it can be clearly resolved whether or not the event occurred and forecasts certainties are expressed as quantitative probabilities. When forecasts are expressed with clarity, then quantitative measures (scoring rules, calibration, discrimination, etc.) can be used to measure forecast accuracy, which in turn can be used to measure the comparative accuracy of different forecasting methods. Unfortunately most real world forecasts are not expressed clearly. This lack of clarity extends to both the description of the forecast event and to the use of vague language to express forecast certainty. It is thus difficult to assess the accuracy of most real world forecasts, and consequently the accuracy the methods used to generate real world forecasts. This paper addresses this deficiency by presenting an approach to measuring the accuracy of imprecise real world forecasts using the same quantitative metrics routinely used to measure the accuracy of well-defined forecasts. To demonstrate applicability, the inferred probability method is applied to measure the accuracy of forecasts in fourteen documents examining complex political domains.

Authors and Affiliations

Paul Lehner, Avra Michelson, Leonard Adelman and Anna Goodman

Keywords

inferred probability imputed probability judgment-based forecasting forecast accuracy imprecise forecasts political forecasting verbal probability probability calibration

On the use of recognition in inferential decision making: An overview of the debate

I describe and discuss the sometimes heated controversy surrounding the recognition heuristic (RH) as a model of inferential decision making. After briefly recapitulating the history of the RH up to its current version,...

Implementation of the Multiple-Measure Maximum Likelihood strategy classification method in R: Addendum to Glöckner (2009) and practical guide for application

One major challenge to behavioral decision research is to identify the cognitive processes underlying judgment and decision making. Glöckner (2009) has argued that, compared to previous methods, process models can be mor...

EP ID EP678029
DOI -
Views 145
Downloads 0