Is This Reliable Enough? Examining Classification Consistency and Accuracy in a Criterion-Referenced Test
Journal Title: International Journal of Assessment Tools in Education - Year 2016, Vol 3, Issue 2
Abstract
One important step for assessing the quality of a test is to examine the reliability of test score interpretation. Which aspect of reliability is the most relevant depends on what type of test it is and how the scores are to be used. For criterion-referenced tests, and in particular certification tests, where students are classified into performance categories, primary focus need not be on the size of error but on the impact of this error on classification. This impact can be described in terms of classification consistency and classification accuracy. In this article selected methods from classical test theory for estimating classification consistency and classification accuracy were applied to the theory part of the Swedish driving licence test, a high-stakes criterion-referenced test which is rarely studied in terms of reliability of classification. The results for this particular test indicated a level of classification consistency that falls slightly short of the recommended level which is why lengthening the test should be considered. More evidence should also be gathered as to whether the placement of the cut-off score is appropriate since this has implications for the validity of classifications.
Authors and Affiliations
Susanne Alger
Keywords
Related Articles
- EP ID EP174034
- DOI 10.21449/ijate.245198
- Views 127
- Downloads 0
Scaling of Ideal Teachers Characteristics with Pairwise Comparison Judgments According to Pre-service Teachers Opinions
In this study, scaling the characteristics that should be found in an ideal teacher according to the pre-service teachers by using the pairwise comparison method was aimed. Thirteen characteristics that an ideal teacher...
Exploring Teachers’ Assessment Practices and Skills
The need for increased use of test results to improve educational outcomes is urgent; yet, there is little understanding in the research literature of practitioners’ knowledge and skills in interpreting and using educati...
The Development of a General Disaster Preparedness Belief Scale Using the Health Belief Model as a Theoretical Framework
The Health Belief Model (HBM) is one of the oldest and most recognized conceptual framework of health behavior and can be applied to disaster preparedness efforts which focus predominantly on human behavior. The study ai...
Are We Measuring Teachers’ Attitudes towards Computers in Detail?: Adaptation of a Questionnaire into Turkish Culture
Teachers’ perceptions of computers play an important role in integrating computers into education. The related literature includes studies developing or adapting a survey instrument in Turkish culture measuring teachers’...
Formative Assessment in Teaching the Macedonian Language (Primary Education in R. Macedonia)
In the Republic of Macedonia, the formative assessment represents a new assessment paradigm that focuses on students as active participants during the teaching process. This paradigm was established in 2007 as part of th...