Criteria for performance evaluation
Journal Title: Judgment and Decision Making - Year 2009, Vol 4, Issue 2
Abstract
Using a cognitive task (mental calculation) and a perceptual-motor task (stylized golf putting), we examined differential proficiency using the CWS index and several other quantitative measures of performance. The CWS index (Weiss & Shanteau, 2003) is a coherence criterion that looks only at internal properties of the data without incorporating an external standard. In Experiment 1, college students (n = 20) carried out 2- and 3-digit addition and multiplication problems under time pressure. In Experiment 2, experienced golfers (n = 12), also college students, putted toward a target from nine different locations. Within each experiment, we analyzed the same responses using different methods. For the arithmetic tasks, accuracy information (mean absolute deviation from the correct answer, MAD) using a coherence criterion was available; for golf, accuracy information using a correspondence criterion (mean deviation from the target, also MAD) was available. We ranked the performances of the participants according to each measure, then compared the orders using Spearman’s rs. For mental calculation, the CWS order correlated moderately (rs =.46) with that of MAD. However, a different coherence criterion, degree of model fit, did not correlate with either CWS or accuracy. For putting, the ranking generated by CWS correlated .68 with that generated by MAD. Consensual answers were also available for both experiments, and the rankings they generated correlated highly with those of MAD. The coherence vs. correspondence distinction did not map well onto criteria for performance evaluation.
Authors and Affiliations
David J. Weiss, Kristin Brennan, Rick Thomas, Alex Kirlik and Sarah M. Miller
Keywords
Related Articles
- EP ID EP677672
- DOI -
- Views 202
- Downloads 0
Validation of the Adult Decision-Making Competence in Slovak students
A study using a high school and college sample (age 18–26) was conducted to validate the Slovak version of the Adult Decision-Making Competence. The results were similar to findings reported by Bruine de Bruin, Parker, a...
Risks deter but pleasures allure: Is pleasure more important?
The pursuit of unhealthy behaviors, such as smoking or binge drinking, not only carries various downside risks, but also provides pleasure. A parsimonious model, used in the literature to explain the decision to pursue a...
Stepwise training supports strategic second-order theory of mind
People model other people’s mental states in order to understand and predict their behavior. Sometimes they model what others think about them as well: “He thinks that I intend to stop.” Such second-order theory of mind...
On the meaning and measurement of maximization
Building on Herbert Simon’s critique of rational choice theory, Schwartz et al. (2002) proposed that when making choices, some individuals — maximizers — search extensively through many alternatives with the goal of maki...
When a risky prospect is valued more than its best possible outcome
In this paper, we document a violation of normative and descriptive models of decision making under risk. In contrast to uncertainty effects found by Gneezy, List and Wu (2006), some subjects in our experiments valued lo...