Is Cross-Marking A Way To Increase Rater Reliability?
Journal Title: International Journal of Langauges' Education and Teaching - Year 2018, Vol 6, Issue 3
Abstract
Most of the error correction research has focused on whether teachers should correct errors in student writing, how they should do it and how deep it should be. Recent research, thus, has mostly focused on pedagogical merits of error correction and its possible benefits for student learning. However, in some particular contexts where graders make multiple scorings on the same paper, not much has been investigated to see if those corrections manipulate other graders or whether the writing teachers’ corrections on students’ papers have a positive or negative impact on the reliability of the scores when raters see the corrections of the other graders on the papers they mark. This study intended to explore whether corrections made by the graders affect the scores of colleagues who are scoring the same papers second time to gain more accurate results and to ensure the rating reliability. To do that, 12 writing teachers graded 20 student essays written by intermediate level English learners. The participants were first asked to grade 10 papers without doing error correction and those papers were re-scored after 3 weeks by the same graders, inter-rater and intra-rater reliability computations were carried out for this set of papers to see the actual reliability levels of the raters under normal circumstances. In the second stage, the graders were asked to score the other 10 papers, but this time they also made error corrections on the papers and after 3 weeks, the same teachers graded the same papers that were corrected by their pair graders. The scores assigned each time to these papers by the same raters, were compared statistically and the effect of error correction was investigated on their scores. In conclusion, the results revealed that error marking and grader comment on writing papers may have a negative effect on raters’ intra-rater reliability levels whereas it could have a positive effect on raters’ inter-rater reliability levels when a pool of raters grade the same papers.
Authors and Affiliations
Murat POLAT
Keywords
Related Articles
- EP ID EP614018
- DOI 10.18298/ijlet.3138
- Views 87
- Downloads 0
An Alternative Method in Teaching Turkish to Foreigners: The Audio-Lingual Method
This study deals with the Auditory-Linguistic Method. The aim of the study is to introduce the philosophical and pedagogical foundations of the Audiovisual Method, a foreign language teaching method. It is considered as...
İkinci/Yabancı Dil Öğretiminde Pedagojik Dil Bilgisinin Yeri
Bir dilin dil bilgisinin amaç ve ele alınış biçimi dil bilgisi türlerini meydana getirmiştir. Dil bilgisi türleri arasında belli başlı türler “Buyurucu Dil Bilgisi” ve “Betimleyici Dil Bilgisi”dir. Bu iki türün yanında b...
INTEGRATING ARTS IN EFL CURRICULA: A FOCUS ON LANGUAGE LISTENING SKILLS
Arts are commonly used in primary and secondary classrooms for learning purposes, but arts integration in higher education curricula could benefit university-level students academically and emotionally as well. Integrati...
Yabancı Dil Kaygısı: Marmara Üniversitesi Yabancı Diller Yüksekokulu Hazırlık Sınıfı Öğrencileri Üzerinde Bir İnceleme
The goal of this research is to investigate foreign language anxiety of the students learning English as a foreign language in preparation classes at Marmara University School of Foreign Language. In accordance with this...
THE FRAGMENTS OF THE TERM “COMON GROUND” DURING ORAL TRANSLATION
We confront previously with the term “Common Ground” as topic, starting point and strategy of many studies. Also, this term provides the sustainability of communication process and underlies the agreement. We could benef...