Is This Reliable Enough? Examining Classification Consistency and Accuracy in a Criterion-Referenced Test
Journal Title: International Journal of Assessment Tools in Education - Year 2016, Vol 3, Issue 2
Abstract
One important step for assessing the quality of a test is to examine the reliability of test score interpretation. Which aspect of reliability is the most relevant depends on what type of test it is and how the scores are to be used. For criterion-referenced tests, and in particular certification tests, where students are classified into performance categories, primary focus need not be on the size of error but on the impact of this error on classification. This impact can be described in terms of classification consistency and classification accuracy. In this article selected methods from classical test theory for estimating classification consistency and classification accuracy were applied to the theory part of the Swedish driving licence test, a high-stakes criterion-referenced test which is rarely studied in terms of reliability of classification. The results for this particular test indicated a level of classification consistency that falls slightly short of the recommended level which is why lengthening the test should be considered. More evidence should also be gathered as to whether the placement of the cut-off score is appropriate since this has implications for the validity of classifications.
Authors and Affiliations
Susanne Alger
Keywords
Related Articles
- EP ID EP174034
- DOI 10.21449/ijate.245198
- Views 117
- Downloads 0
Middle School Mathematics Teachers' Opinions on Feedback
During instruction, providing feedbacks improves students’ academic achievements as well as motivates them to actively engage in lesson activities. Feedback is very important for teaching. Feedback is not only a function...
Determination of Relationship Between the Empathic Tendency Levels and Thinking Styles of Preschool Teacher Candidates
The main objective of this study is to determine the relationship between the empathic tendency levels and thinking styles of preschool teacher candidates. In this study that was patterned with the relational survey mode...
Development of the rubric self-efficacy scale
The purpose of this study is to develop a valid and reliable measurement tool determining teachers’ self-efficacy regarding rubrics. Especially in educational environments, rubrics are measurement tools used in the asses...
A Comparison of Logistic Regression Models for Dif Detection in Polytomous Items: The Effect of Small Sample Sizes and Non-Normality of Ability Distributions
This study investigated the effectiveness of logistic regression models to detect uniform and non-uniform DIF in polytomous items across small sample sizes and non-normality of ability distributions. A simulation study w...
The Validity and Reliability Study of Revised School Climate Teacher Survey’s Turkish Version
It is aimed to adapt Revised School Climate Teacher Survey (RSCTS) which is developed with a character education perspective to Turkish and assess its psychometrics properties in this study. This study is an instrument a...