Is This Reliable Enough? Examining Classification Consistency and Accuracy in a Criterion-Referenced Test
Journal Title: International Journal of Assessment Tools in Education - Year 2016, Vol 3, Issue 2
Abstract
One important step for assessing the quality of a test is to examine the reliability of test score interpretation. Which aspect of reliability is the most relevant depends on what type of test it is and how the scores are to be used. For criterion-referenced tests, and in particular certification tests, where students are classified into performance categories, primary focus need not be on the size of error but on the impact of this error on classification. This impact can be described in terms of classification consistency and classification accuracy. In this article selected methods from classical test theory for estimating classification consistency and classification accuracy were applied to the theory part of the Swedish driving licence test, a high-stakes criterion-referenced test which is rarely studied in terms of reliability of classification. The results for this particular test indicated a level of classification consistency that falls slightly short of the recommended level which is why lengthening the test should be considered. More evidence should also be gathered as to whether the placement of the cut-off score is appropriate since this has implications for the validity of classifications.
Authors and Affiliations
Susanne Alger
Keywords
Related Articles
- EP ID EP174034
- DOI 10.21449/ijate.245198
- Views 133
- Downloads 0
Evaluating the Comparability of PPT and CBT by Implementing the Compulsory Islamic Culture Course Test in Jordan University
Study aims to determine whether the university students' scores in the compulsory Islamic culture course test on a selected sample differ across the paper-and pencil test (PPT) & computer-based test (CBT) versions, and t...
Higher Education End-of-Course Evaluations: Assessing the Psychometric Properties Utilizing Exploratory Factor Analysis and Rasch Modeling Approaches
This paper offers a critical assessment of the psychometric properties of a standard higher education end-of-course evaluation. Using both exploratory factor analysis (EFA) and Rasch modeling, the authors investigate th...
The Use of Academic Portfolio in the Learning and Assessment of Physics Students from a Singapore Private College
The purpose of this perspective paper is to examine the use of portfolios in the teaching and learning of physics in a Singapore private college. The paper starts with a short introduction of the types of students and th...
Investigation of 9th Grade High School Students’ Attitudes towards Science Course
In this study, ninth grade students’ attitudes towards science were investigated in terms of self-regulation strategies, motivational beliefs and gender variables. The sample of this study includes 322 male and 296 femal...
Developing The Moral Competencies of Accounting Students: A Case Study of International Islamic University Malaysia (IIUM)
Two decades of financial scandals have seriously damaged the credibility of accountants as guardians of financial information. To repair this credibility, the Malaysian government released a blueprint that mandated Malay...