Criteria for performance evaluation
Journal Title: Judgment and Decision Making - Year 2009, Vol 4, Issue 2
Abstract
Using a cognitive task (mental calculation) and a perceptual-motor task (stylized golf putting), we examined differential proficiency using the CWS index and several other quantitative measures of performance. The CWS index (Weiss & Shanteau, 2003) is a coherence criterion that looks only at internal properties of the data without incorporating an external standard. In Experiment 1, college students (n = 20) carried out 2- and 3-digit addition and multiplication problems under time pressure. In Experiment 2, experienced golfers (n = 12), also college students, putted toward a target from nine different locations. Within each experiment, we analyzed the same responses using different methods. For the arithmetic tasks, accuracy information (mean absolute deviation from the correct answer, MAD) using a coherence criterion was available; for golf, accuracy information using a correspondence criterion (mean deviation from the target, also MAD) was available. We ranked the performances of the participants according to each measure, then compared the orders using Spearman’s rs. For mental calculation, the CWS order correlated moderately (rs =.46) with that of MAD. However, a different coherence criterion, degree of model fit, did not correlate with either CWS or accuracy. For putting, the ranking generated by CWS correlated .68 with that generated by MAD. Consensual answers were also available for both experiments, and the rankings they generated correlated highly with those of MAD. The coherence vs. correspondence distinction did not map well onto criteria for performance evaluation.
Authors and Affiliations
David J. Weiss, Kristin Brennan, Rick Thomas, Alex Kirlik and Sarah M. Miller
Keywords
Related Articles
- EP ID EP677672
- DOI -
- Views 198
- Downloads 0
How do defendants choose their trial court? Evidence for a heuristic processing account
In jurisdictions with two or more tiers of criminal courts, some defendants can choose the type of trial court to be tried in. This may involve a trade-off between the probability of acquittal/conviction and the estimate...
Moral emotions as determinants of third-party punishment: Anger, guilt, and the functions of altruistic sanctions
Third-party punishment has recently received attention as an explanation for human altruism. Feelings of anger in response to norm violations are assumed to motivate third-party sanctions, yet there is only sparse and in...
Deception and price in a market with asymmetric information
In markets with asymmetric information, only sellers have knowledge about the quality of goods. Sellers may of course make a declaration of the quality, but unless there are sanctions imposed on false declarations or rep...
The tide that lifts all focal boats: Asymmetric predictions of ascent and descent in rankings
In six studies, we find evidence for an upward mobility bias, or a tendency to predict that a rise in ranking is more likely than a decline, even in domains where motivation or intention to rise play no role. Although pe...
Effects of main actor, outcome and affect on biased braking speed judgments
Subjects who judged speed in a driving scenario overestimated how fast they could decelerate when speeding compared to when keeping within the speed limit (Svenson, 2009). The purpose of the present studies were to repli...