Analyzing randomized controlled interventions: Three notes for applied linguists
Journal Title: Studies in Second Language Learning and Teaching - Year 2015, Vol 5, Issue 1
Abstract
I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and (c) using traditional significance tests to analyze interventions in which whole groups were assigned to the conditions (cluster randomization). The first practice is labeled superfluous, and taking full advantage of important covariates regardless of balance is recommended. The second is needlessly complicated, and analysis of covariance is recommended as a more powerful alternative. The third produces dramatic inferential errors, which are largely, though not entirely, avoided when mixed-effects modeling is used. This discussion is geared towards applied linguists who need to design, analyze, or assess intervention studies or other randomized controlled trials. Statistical formalism is kept to a minimum throughout.
Authors and Affiliations
Jan Vanhove
Keywords
Related Articles
- EP ID EP256517
- DOI 10.14746/ssllt.2015.5.1.7
- Views 155
- Downloads 0
Revising the Metacognitive Awareness of Reading Strategies Inventory (MARSI) and testing for factorial invariance
In this study, we revised the Metacognitive Awareness of Reading Strategies Inventory (MARSI), a self-report instrument designed to assess students’ awareness of reading strategies when reading school-rela...
The use of articles in L2 English: A phraseological perspective
While it is a well-known fact that speakers of article-less mother tongues, such as Polish, experience problems with articles in English, this study seeks to investigate the problem from a different perspective. Namely,...
Motivation and demotivation over two years: A case study of English language learners in Japan
This paper is about four Japanese university students majoring in international studies, who participated in a two-year study examining changes in their motivation. Using monthly interviews and a 29-item questionnaire on...
Expressing authorial self in research articles written by Polish and English native-speaker writers: A corpus-based study
This cross-linguistic and cross-cultural, corpus-based study explores the notion of writer identity expressed through self-reference. The study examines how writers from two cultural regions – Polish and Anglo-American –...
Parental visions of their children’s future as a motivator for an early start in a foreign language
This paper reports on the qualitative part of a project investigating parental educational aspirations as manifested by enrolling their children (aged 3-6) into very early L2 instruction. The concept of educational aspir...