Methodological notes on model comparisons and strategy classification: A falsificationist proposition
Journal Title: Judgment and Decision Making - Year 2011, Vol 6, Issue 8
Abstract
Taking a falsificationist perspective, the present paper identifies two major shortcomings of existing approaches to comparative model evaluations in general and strategy classifications in particular. These are (1) failure to consider systematic error and (2) neglect of global model fit. Using adherence measures to evaluate competing models implicitly makes the unrealistic assumption that the error associated with the model predictions is entirely random. By means of simple schematic examples, we show that failure to discriminate between systematic and random error seriously undermines this approach to model evaluation. Second, approaches that treat random versus systematic error appropriately usually rely on relative model fit to infer which model or strategy most likely generated the data. However, the model comparatively yielding the best fit may still be invalid. We demonstrate that taking for granted the vital requirement that a model by itself should adequately describe the data can easily lead to flawed conclusions. Thus, prior to considering the relative discrepancy of competing models, it is necessary to assess their absolute fit and thus, again, attempt falsification. Finally, the scientific value of model fit is discussed from a broader perspective.
Authors and Affiliations
Morten Moshagen and Benjamin E. Hilbig
Keywords
Related Articles
- EP ID EP677864
- DOI -
- Views 143
- Downloads 0
Reluctant altruism and peer pressure in charitable giving
Subjects donate individually (control group) or in pairs (treatment group). Those in pairs reveal their donation decision to each other. Average donations in the treatment group are significantly higher than in the contr...
Wronging past rights: The sunk cost bias distorts moral judgment
When people have invested resources into an endeavor, they typically persist in it, even when it becomes obvious that it will fail. Here we show this bias extends to people’s moral decision-making. Across two preregister...
Direct and indirect effects of pathological gambling on risk attitudes
We study individual decision making in a lottery-choice task performed by three different populations: gamblers under psychological treatment (äddicts"), gamblers' spouses ("victims"), and people who are neither gamblers...
Anticipated and experienced emotions in environmental risk perception
Affective forecasting with respect to two environmental risks (ozone depletion, air pollution) was investigated by studying tourists who travelled to either Australia or Bangkok and were thus confronted with one of these...
American’s desire for less wealth inequality does not depend on how you ask them
A large body of survey research offers evidence that citizens are not always fully aware of the economic and political realities in their respective countries. Norton and Ariely (2011) extended this research to the domai...