Pair-wise comparisons of multiple models
Journal Title: Judgment and Decision Making - Year 2011, Vol 6, Issue 8
Abstract
Often research in judgment and decision making requires comparison of multiple competing models. Researchers invoke global measures such as the rate of correct predictions or the sum of squared (or absolute) deviations of the various models as part of this evaluation process. Reliance on such measures hides the (often very high) level of agreement between the predictions of the various models and does not highlight properly the relative performance of the competing models in those critical cases where they make distinct predictions. To address this important problem we propose the use of pair-wise comparisons of models to produce more informative and targeted comparisons of their performance, and we illustrate this procedure with data from two recently published papers. We use Multidimensional Scaling of these comparisons to map the competing models. We also demonstrate how intransitive cycles of pair-wise model performance can signal that certain models perform better for a given subset of decision problems.
Authors and Affiliations
Stephen B. Broomell, Budescu, David V. and Han-Hui Por
Keywords
Related Articles
- EP ID EP677865
- DOI -
- Views 142
- Downloads 0
How many calories were in those hamburgers again? Distribution density biases recall of attribute values
Decisions that consumers make often rest on evaluations of attributes, such as how large, expensive, good, or fattening an option seems. Extant research has demonstrated that these evaluations in turn depend upon the rec...
Glucose promotes controlled processing: Matching, maximizing, and root beer
Participants drank either regular root beer or sugar-free diet root beer before working on a probability-learning task in which they tried to predict which of two events would occur on each of 200 trials. One event (E1)...
Reducing the impact bias in judgments of post-decisional affect: Distraction or task interference.?
People overestimate their affective reactions to future events and decisions — a phenomenon that has been termed “impact bias.” Evidence suggests that completing a diary detailing events contemporaneous with the focal on...
Can asymmetric subjective opportunity cost effect explain impatience in intertemporal choice? A replication study
In “The value of nothing: asymmetric attention to opportunity costs drives intertemporal decision making” Read, Olivola and Hardisty (2017) proposed an asymmetric subjective opportunity cost (ASOC) effect to explain and...
Does moving from war zone change emotions and risk perceptions? A field study of Israeli students
The current field study uses data collected after the 2009 war between Israel and the Hamas militias in the Gaza Strip ended. The study compares recalled emotions and perceived risks among two groups of students, all of...