Analyzing randomized controlled interventions: Three notes for applied linguists
Journal Title: Studies in Second Language Learning and Teaching - Year 2015, Vol 5, Issue 1
Abstract
I discuss three common practices that obfuscate or invalidate the statistical analysis of randomized controlled interventions in applied linguistics. These are (a) checking whether randomization produced groups that are balanced on a number of possibly relevant covariates, (b) using repeated measures ANOVA to analyze pretest-posttest designs, and (c) using traditional significance tests to analyze interventions in which whole groups were assigned to the conditions (cluster randomization). The first practice is labeled superfluous, and taking full advantage of important covariates regardless of balance is recommended. The second is needlessly complicated, and analysis of covariance is recommended as a more powerful alternative. The third produces dramatic inferential errors, which are largely, though not entirely, avoided when mixed-effects modeling is used. This discussion is geared towards applied linguists who need to design, analyze, or assess intervention studies or other randomized controlled trials. Statistical formalism is kept to a minimum throughout.
Authors and Affiliations
Jan Vanhove
Keywords
Related Articles
- EP ID EP256517
- DOI 10.14746/ssllt.2015.5.1.7
- Views 150
- Downloads 0
Beyond instructed L2 grammar acquisition: Theoretical insights and pedagogical considerations about the role of prior language knowledge
The prior language knowledge of learners for whom the target language is not the first foreign language poses a different starting learning situation that should merit pedagogical attention. The present paper seeks to co...
“This language still motivates me!” Advanced language students and their L2 motivation
The article focuses on written narratives of 51 Finnish university students who study German, Swedish or French as their major or one of their minors at an advanced level. The study aims to find what keeps these students...
A study of Taiwanese university students’ English use, learning goals and attitudes toward English as a lingua franca
This study investigates Taiwanese university students’ experience of English use, aims of learning English and attitudes toward English as a lingua franca (ELF). The notion of ELF has been researched in the field of Engl...
“Did we learn English or what?”: A study abroad student in the UK carrying and crossing boundaries in out-of-class communication
Language educators in many parts of the world are torn between preparing language learners to pass language proficiency tests and trying to let their classrooms reflect the messiness of out-of-class communication. Becaus...
The L2 motivational self system: A meta-analysis
This article reports the first meta-analysis of the L2 motivational self system (Dörnyei, 2005, 2009). A total of 32 research reports, involving 39 unique samples and 32,078 language learners, were meta-analyzed. The res...