Assessment Methods and Results
Senta Goertler & Adam Gacs
Michigan State University
There is no question that online education has been on the rise (Allen & Seaman, 2014), how-ever, language instructors have been especially cautious about embracing online delivery formats(Arnold, 2007; Kessler, 2010; Reinders & White, 2016). Typical worries about language education online include increased time commitment for students and teachers (Kraemer, 2008), testing se-curity (Chenoweth, Jones, & Tucker, 2006), learning outcomes (Timms, 2017), and the concern
that it is intended to save costs and eliminate teachers (Blake, 2001, 2008). One of the greatest benefits of online education is the flexibility it offers in time and place of instruction, which makes language education possible for those who would otherwise lack access (Hampel & Hauck, 2004;
Meskill & Anthony, 2015). As argued elsewhere (Goertler, forthcoming), online education is here to stay and educators, regardless of their opinions about it, must become conversant with its pos- sibilities and limitations, advantages and disadvantages. Here we hope to add to the knowledge base about online instruction by focusing specifically on assessment, namely (1) by investigating
best practices in assessment in online language courses and (2) by comparing learning outcomes in online courses with face-to-face (F2F) courses. We discuss both previous research as well as the practices and results at Michigan State University (MSU), our own institution.
Background
Definitions and Trends
Now that technology has become more prevalent in educational circles in general and lan-guage learning in particular, there are many technology-mediated delivery formats from which to choose: from technology-enhanced instruction, where a F2F class employs online technolo- gies and/or mobile devices (Plonsky & Ziegler, 2016), to blended or flipped courses, where some class time is replaced with or shifted to online activities (Hojnacki, 2018; Hung, 2015),
to regular courses offered completely online, to massive open online courses (MOOCs) (Dixon& Thomas, 2015). We concentrate on language courses in which more than 80% of the in-structional time is online and which may include both asynchronous and synchronous instruc-tion and compare them to their counterparts in a F2F format.
When offering online and F2F sections of the same course, one has to ensure that despite the different demands of the delivery formats, the educational objectives and prospective out-comes remain consistent. In order to achieve this, curriculum developers and language educators must consider a number of factors: what media and platforms are available to them, how to offer interactive online components, what instruction can be done asynchronously and what must be done synchronously, and which types of students and teachers are more and less likely to succeed in the online format (e.g., more autonomous behaviors are often required from learn- ers and skill in detailed planning and an ability to move into the background in real time from
teachers). Several informative handbooks about online teaching have been written (e.g., Meskill & Anthony, 2015) and thus we chose to focus here instead on issues of assessment only.
As mentioned in the introduction, online education in general has grown rapidly (Allen & Seaman, 2014). Not only has the number of offerings increased over time, so have improve- ments in quality and format. Some programs, such as ours, reduce class size for online courses to allow for more frequent and meaningful interactions, while others maximize the format to reach the broadest audience possible through MOOCs, such as at the University of Pennsylvania (Dixon & Fuchs, 2015). Survey data collected by Murphy-Judy and Johnshoy (2017) also sug-gest an increase in the number of online courses in K-16 world language curricula, with most online courses in basic language programs. MSU started moving one section of the German basic language program online in 2014; some other institutions have had a much longer tradi- tion, for example Carnegie Mellon (e.g., Chenoweth & Murday, 2003). Online education also allows for cross-institutional collaboration in teaching languages, especially less commonly taught languages, where courses and materials may be shared (Alosh, 2001; Goertler, forthcoming).
Best Practices in Assessing Language Learning Online
When one considers offering online courses, it is important not only to follow best practices in design and implementation but also in assessment, starting with a thorough needs analysis (Gonzalez-Lloret, 2014; Gruba, Cardenas-Claros, Suvorov, & Rick, 2016). Once a design team understands the student population’s current and desired language and technology competen-cies as well as the instructional context, a backward design approach is especially suited for on-line course development and decisions about student assessments. Backward curriculum design begins with the end, so to speak, with the learning goals and learning outcomes. From these,
one moves backward to develop the tasks and materials that facilitate such learning (Wiggins & McTighe, 1998). While many F2F and online courses use a backward design approach, it is more obviously needed in online courses, where adjustments are harder to make during the course implementation, and the course design should be complete prior to its implementation.
In the case of language instruction, the desired proficiency levels as well as linguistic goal tasks are determined first. In second-year German at MSU, the goals are derived from the compe-tencies listed for the ACTFL Intermediate range (Goertler, Kraemer, & Schenker, 2016).
Next in the development process, designers must decide how to measure student progress toward those goals in a valid, reliable, feasible, and informative way (Gruba et al., 2016).
In an online course, students need opportunities for frequent formative assessment in the different modes of communication to inform instructors and themselves about their progress and to receive feedback. There are several challenges in online assessment. First, with online assessment, course designers must anticipate or determine the technology skills and needs of students, as students must
be evaluated on their language skills and not their computer skills. Hence the aforementioned need for a thorough needs analysis. Assessment design must attend to students with limited technology skills or low comfort levels with technology tools as they may inadvertently lower scores on per-formance assessments. Designers can help mitigate those dangers, however, with practice tests, tu-torials, and models of successful task completions. Second, the online format also poses new questions in assessment regarding the protection of students’ privacy and intellectual property (Zink,2018). For example, depending on the platforms used for assessment, the content may be owned by a third party and/or analytics may be performed on the data. Third, the technology format re-quires new and different accommodations for learners with disabilities (Gonzalez-Lloret, 2014).
Universal Design for Learning principles (CAST, 2018) may not only aid students who request spe- cific accommodations but may benefit all course users by providing multiple means to receive input and to communicate. And finally, test security is another area that requires careful consideration in the online format. If a course is offered in both online and F2F formats, articulation between F2F and online sections is crucial for both curriculum design and assessment procedures.
GOERTLER/GACS: ASSESSMENT IN ONLINE GERMAN 157
