Criteria for performance evaluation
Journal Title: Judgment and Decision Making - Year 2009, Vol 4, Issue 2
Abstract
Using a cognitive task (mental calculation) and a perceptual-motor task (stylized golf putting), we examined differential proficiency using the CWS index and several other quantitative measures of performance. The CWS index (Weiss & Shanteau, 2003) is a coherence criterion that looks only at internal properties of the data without incorporating an external standard. In Experiment 1, college students (n = 20) carried out 2- and 3-digit addition and multiplication problems under time pressure. In Experiment 2, experienced golfers (n = 12), also college students, putted toward a target from nine different locations. Within each experiment, we analyzed the same responses using different methods. For the arithmetic tasks, accuracy information (mean absolute deviation from the correct answer, MAD) using a coherence criterion was available; for golf, accuracy information using a correspondence criterion (mean deviation from the target, also MAD) was available. We ranked the performances of the participants according to each measure, then compared the orders using Spearman’s rs. For mental calculation, the CWS order correlated moderately (rs =.46) with that of MAD. However, a different coherence criterion, degree of model fit, did not correlate with either CWS or accuracy. For putting, the ranking generated by CWS correlated .68 with that generated by MAD. Consensual answers were also available for both experiments, and the rankings they generated correlated highly with those of MAD. The coherence vs. correspondence distinction did not map well onto criteria for performance evaluation.
Authors and Affiliations
David J. Weiss, Kristin Brennan, Rick Thomas, Alex Kirlik and Sarah M. Miller
Keywords
Related Articles
- EP ID EP677672
- DOI -
- Views 197
- Downloads 0
Why choose wisely if you have already paid? Sunk costs elicit stochastic dominance violations
Sunk costs have been known to elicit violations of expected utility theory, in particular, the independence or cancellation axiom. Separately, violations of the stochastic dominance principle have been demonstrated in va...
How should we think about Americans’ beliefs about economic mobility?
Recent evidence suggests that Americans’ beliefs about upward mobility are overly optimistic. Davidai & Gilovich (2015a), Kraus & Tan (2015), and Kraus (2015) all found that people overestimate the likelihood that a pers...
Analytic atheism: A cross-culturally weak and fickle phenomenon?
Religious belief is a topic of longstanding interest to psychological science, but the psychology of religious disbelief is a relative newcomer. One prominently discussed model is analytic atheism, wherein cognitive refl...
Biases in choices about fairness: Psychology and economic inequality
This paper investigates choices about “distributional fairness” (sometimes called “distributive justice”), i.e., selection of the proper way for resources to be distributed in group. The study finds evidence that several...
Action orientation, consistency and feelings of regret
Previous research has demonstrated that consistency between people's behavior and their dispositions has predictive validity for judgments of regret. Research has also shown that differences in the personality variable o...