Is Cross-Marking A Way To Increase Rater Reliability?
Journal Title: International Journal of Langauges' Education and Teaching - Year 2018, Vol 6, Issue 3
Abstract
Most of the error correction research has focused on whether teachers should correct errors in student writing, how they should do it and how deep it should be. Recent research, thus, has mostly focused on pedagogical merits of error correction and its possible benefits for student learning. However, in some particular contexts where graders make multiple scorings on the same paper, not much has been investigated to see if those corrections manipulate other graders or whether the writing teachers’ corrections on students’ papers have a positive or negative impact on the reliability of the scores when raters see the corrections of the other graders on the papers they mark. This study intended to explore whether corrections made by the graders affect the scores of colleagues who are scoring the same papers second time to gain more accurate results and to ensure the rating reliability. To do that, 12 writing teachers graded 20 student essays written by intermediate level English learners. The participants were first asked to grade 10 papers without doing error correction and those papers were re-scored after 3 weeks by the same graders, inter-rater and intra-rater reliability computations were carried out for this set of papers to see the actual reliability levels of the raters under normal circumstances. In the second stage, the graders were asked to score the other 10 papers, but this time they also made error corrections on the papers and after 3 weeks, the same teachers graded the same papers that were corrected by their pair graders. The scores assigned each time to these papers by the same raters, were compared statistically and the effect of error correction was investigated on their scores. In conclusion, the results revealed that error marking and grader comment on writing papers may have a negative effect on raters’ intra-rater reliability levels whereas it could have a positive effect on raters’ inter-rater reliability levels when a pool of raters grade the same papers.
Authors and Affiliations
Murat POLAT
Keywords
Related Articles
- EP ID EP614018
- DOI 10.18298/ijlet.3138
- Views 91
- Downloads 0
Teachers’ and Students’ Sense of Teachers’ Emotional and Professional Identity
Emotions are integrated into every sphere of people’s lives since they are one of the core elements of human beings. Being in a profession wherein human being is at the very center, teachers experience various feelings a...
SEMIOTIC AND SEMANTIC ANALYSIS OF TREE CONCEPT IN DEDE KORKUT BOOK
The study of semiotics and semantics in Linguistic starts with Ferdinand de Saussure in the modern sense. Semiotics, known also as semiology, has the answer to the question how a sign that is present in the mind come int...
Değerler Eğitimi Açısından Aysel Gürmen’in 8-12 Yaş Aralığındaki Çocuklar İçin Yazdığı Eserlere Bir Bakış
Milletlerin alışkanlıkları, inançları, hayata bakış ve değerlendiriş tarzları, olaylar karşısında vermiş oldukları tepkiler; kültürün içerisinde değerlendirilmektedir. Birbirini takip eden nesiller arasında oluşturulaca...
THE USE OF TRAVEL JOURNALS IN FOREIGN LANGUAGE EDUCATION
In this work, the definition of travel journals as a genre of literature will be made in light of the enunciation theory from linguistics and semiotics, and perception theory from semiotics which is inspired by phenomeno...
A PEDAGOGICAL CRITICAL REVIEW OF ONLINE LEARNING SYSTEM
E-learning which have various shapes such as blog, classroom learning which is facilitated the World Wide Web; a mix of online instruction and meeting the class known as additional models or hybrid; or the full online ex...