Methodological notes on model comparisons and strategy classification: A falsificationist proposition
Journal Title: Judgment and Decision Making - Year 2011, Vol 6, Issue 8
Abstract
Taking a falsificationist perspective, the present paper identifies two major shortcomings of existing approaches to comparative model evaluations in general and strategy classifications in particular. These are (1) failure to consider systematic error and (2) neglect of global model fit. Using adherence measures to evaluate competing models implicitly makes the unrealistic assumption that the error associated with the model predictions is entirely random. By means of simple schematic examples, we show that failure to discriminate between systematic and random error seriously undermines this approach to model evaluation. Second, approaches that treat random versus systematic error appropriately usually rely on relative model fit to infer which model or strategy most likely generated the data. However, the model comparatively yielding the best fit may still be invalid. We demonstrate that taking for granted the vital requirement that a model by itself should adequately describe the data can easily lead to flawed conclusions. Thus, prior to considering the relative discrepancy of competing models, it is necessary to assess their absolute fit and thus, again, attempt falsification. Finally, the scientific value of model fit is discussed from a broader perspective.
Authors and Affiliations
Morten Moshagen and Benjamin E. Hilbig
Keywords
Related Articles
- EP ID EP677864
- DOI -
- Views 125
- Downloads 0
Liberal-conservative differences in inclusion-exclusion strategy choice
Inclusion and exclusion strategies for allocation of scarce goods involve different processes. The conditions under which one strategy is chosen in favor of the other, however, have not been fully explicated. In the pres...
Dual processes and moral conflict: Evidence for deontological reasoners’ intuitive utilitarian sensitivity
The prominent dual process model of moral cognition suggests that reasoners intuitively detect that harming others is wrong (deontological System-1 morality) but have to engage in demanding deliberation to realize that h...
Using hierarchical Bayesian methods to examine the tools of decision-making
Hierarchical Bayesian methods offer a principled and comprehensive way to relate psychological models to data. Here we use them to model the patterns of information search, stopping and deciding in a simulated binary com...
Cross-cultural support for a link between analytic thinking and disbelief in God: Evidence from India and the United Kingdom
A substantial body of evidence suggests that favoring reason over intuition (employing an analytic cognitive style) is associated with reduced belief in God. In the current work, we address outstanding issues in this lit...
Precise models deserve precise measures: A methodological dissection
The recognition heuristic (RH) — which predicts non-compensatory reliance on recognition in comparative judgments — has attracted much research and some disagreement, at times. Most studies have dealt with whether or und...