Evaluating Online Language-Learning Materials: A CALL Evaluation Checklist

Abstract

Teachers are faced with a vast number of online materials of varying quality. This assessment develops a CALL evaluation checklist - with criteria such as teacher fit, language learning potential and practicality - and demonstrates it by reviewing real websites for exam training.

Rationale

Introduction

In order to use relevant technological recourses in the classroom, teachers need to be able to evaluate the online materials they want to use based on relevance for their current students as one website might be useful for one class but may have no relevance for another (Hubbard, 2019). Creating a suitable CALL evaluation checklist will guide and support teachers to identify compatible resources from the increasingly large number of materials online. This evaluation form will mainly focus on international students from European countries who have come to Australia to study English in an English as a Second Language (ESL) environment. The students have enrolled in a ten-week Cambridge Advanced in English (CAE) exam course. To receive a C1 (advanced level) certificate in the Common European Framework of Reference (CEFR), students need to pass the exam at the end of the course. Alongside their course, the students will need to practice these exam skills in their own time using CALL technology. This evaluation will focus on the reading component of the exam as learners struggle to find reading materials that are compatible with the one in the exam to practice on their own.

Specific Evaluation Criteria

In this evaluation, a variety of checklists have been adapted and implemented, so that teachers can use it easily to ensure student satisfaction.

Figure 1. An adapted version of Hubbard’s (2006) Software Evaluation Framework to fit this evaluation Operational Preview. Hubbard (2019) stresses the importance of looking at materials from both ends as both teacher and student have to navigate the resources from different perspectives. The teacher needs to be able to identify the relevance of using and introducing the product to learners, and the students need to be able to use the product without a teacher present. In order to accomplish that, the product needs to be attractive, easy to use, and the students need to see the relevance in correlation to their exam course (Hubbard, 2019). The websites need to be without fees, signing up, nor advertisement to avoid difficulty and frustration, but also easy to navigate (Son, 2005). This is why Son’s (2005) selection criteria is used in this section, as the questions are general and straightforward for teachers to use. This part of the evaluation gives teachers a simple 10-step overview of the software focusing on the main criteria as a user to get an understanding of how the software operates as a whole before going into detail. The user needs at least 6 out of 10 points to continue completing the form. Teacher Fit and Language Potential. This part measures the compatibility of the software in alliance with the methods of a teacher to deduce if it is Learner Fit (i.e., does the content correspond with the learners’ needs). This will encourage teachers to evaluate the software objectively and understand the necessity of the program in their course (Murray&Barnes, 1998). Task Appropriateness and Implementation. This part is adapted and reformulated from Chapelle (2001) to generate detailed information regarding the effectiveness of reading tasks and their technical features. For example, to ensure a reading task is Teacher and Learner Fit, Chapelle (2001) suggests that next to technical features online reading tests should have open-ended comprehension questions as these items challenge students at an academic level as it tests learners’ linguistic output in form of short phrases and sentences. Hence a division between: General and technical features. A checklist of Boolean as the statements are straightforward. Pedagogical features. Measured by the 5-point Likert scale to ensure the appropriateness of the software in detail due to its importance to learners as they must show a certain degree of understanding of the reading. Evaluation Results. Finally, an overarching summary showing whether or not the software is compatible in use (Hubbard, 2006) for learners preparing for the CAE reading exam.

Review of the selected websites

Overall, the websites selected were Good-Appropriate (69-79%) or Excellent-Highly (80-100%) recommended for use. However, what did stand out was the fact that Cambridge English’s website scored the lowest whilst made for Cambridge exam training purposes, whereas British Council’s website is made for General English purposes but managed to score higher due to its up-to-date, innovative, and interactive website.

Conclusion

The main aim of this evaluation is to assess language websites for CAE exam training in reading skills. The evaluation form was created based on the pre-existing literature on the topic and my preferences in evaluating websites for teaching purposes without having a teacher present. Three websites were evaluated with the created form and the results did not match the assumptions of the websites as one of which was designed for CAE exam skills training. This shows the importance of using a checklist to ensure the websites are Teacher and Learner Fit before suggesting them to learners or implementing them in classroom settings. (Wordcount: 810)

References

Chapelle, C. (2001). Computer applications in second language acquisition: foundations for teaching, testing and research. Cambridge: Cambridge University Press. Hubbard, P. (2006). Evaluating CALL Software, in Ducate, L. and Arnold, N. Calling on CALL: from theory and research to new directions in foreign language teaching. San Marcos: Texas (CALICO).

Hubbard, P. (2019). An Invitation to CALL Foundations of Computer-Assisted Language Learning. Stanford University.

Murray, L., & Barnes, A. (1998). Beyond the “wow” factor - Evaluating multimedia language learning software from a pedagogical viewpoint. System, 26(2), 249-259. doi:http://dx.doi.org/10.1016/S0346-251X(98)00008-6

Son, Jeong-Bae (2005) Exploring and evaluating language learning Web sites. In: Enhancing

← Back to Papers