This is a short critique on Aldukhayel, D. (2021). Vlogs in L2 listening: EFL learners’ and teachers’ perceptions. Computer Assisted Language Learning, 34(8), 1085–1104. https://doi.org/10.1080/09588221.2019.1658608

📖 The paper

Arguing for the higher diversity of online resources that may better fit students’ personal preferences than textbook materials, the selected paper reports a mixed-method study investigating how EFL learners and practitioners perceive the role of vlogs in L2 listening. The author elicited the attitudes of 389 students and their teachers towards a four-week programme with vlogs used in the English classes. The results suggest a generally positive perception of vlogs. The present essay will evaluate the strengths and limitations of the selected paper.

❌ Is it trustworthy?

Rationale and Research Design

After reviewing the existing studies about the benefits of using movies in language learning, it is suggested that the learners could benefit similarly (if not more) from watching the whole vlog (shorter than 15 minutes) compared with a short excerpt from a movie. However, the paper did not discuss whether the styles of vlogs can be unique in the L2 learning process, which may neglect the various types of L2 discourses. Unlike movies, vlogs were recorded as a single producer (vlogger) informally speaking to the camera 📷, which represents a particular function of text (mode) and speaker-listener relationship (tenor; Halliday & Matthiessen, 2014). In this respect, it is arguable how the monologic vlog can contribute to pragmatic and communicative competence because nearly no interpersonal reactions were presented. Therefore, rather than arbitrarily test the superiority of vlogs over textbook materials, it is better to test how L2 acquisition can be boosted by a spectrum of materials, including but not limited to movies (interactive and prepared), vlogs (monologic and sometimes spontaneous), interviews (interactive and sometimes spontaneous), TED talks (monologic and prepared), and even video game streaming (interactive and spontaneous).

Besides, the selected article did provide another more tenable rationale, that is, the gradual and long-term development of autonomous learning behaviours after school (i.e., extensive viewing of vlogs; p. 2). Nevertheless, although highlighted throughout the paper, this promising outcome of vlog-based learning was not even tested in the study. Similarly, despite identifying the advantage of interacting with the vlogging community via online comments (p. 4), the design per se did not extend beyond the use of vlogs solely as a source of input in the classroom setting. Even if we narrow the scope only to the classroom context, the present study still suffers from low validity. Firstly, the selected research investigated the perceptions of utilising vlogs without correlating it to the final L2 achievement. It is possible that the L2 learners 'enjoy' the materials but do not improve afterwards. Secondly, all the participants were in the experiment group (i.e., there are no control groups), which means that the effectiveness of vlog watching cannot be easily compared with that of other materials. Thirdly, concerning the reporting of research design, while the criteria of vlog selection were described, it is not replicable enough because little is known about the details such as the range of topics, the proportion of learnt/unlearnt lexicons, and variations of English used by the vlogger.

📋 Results and Discussion

The quantitative part is limited by the questionnaire’s weak construct validity and psychometric property. The first issue about the instrument is that most of the items only reveal the attitude towards vlogs but not how they could be located in a wide range of learning materials. Regarding this, when the students rated highest in the item ‘Vlogs can enhance my English listening skill’, it can also be the case that the same respondents will agree that ‘Textbook materials can also enhance my[their] English listening skill’. In addition, the items are all positively worded. As the participants in a survey-based study would prefer to give favourable answers (Dörnyei & Taguchi, 2009), it is unsurprising to result in a generally positive perception based on such a questionnaire, which even invalidates the Cronbach’s Alpha index reported in the paper. If reverse coding could be applied, this study could significantly enhance the validity of this instrument. In the qualitative part, as it would like to ‘triangulate’ (p. 7) the quantitative results, the data collection needs to be even more cautious because the researchers’ pre-registered expectations 👩‍🎓‍ may negatively affect the results. For example, the students were asked ‘What was interesting and helpful about vlogs?’ (p. 7) which presupposed that all of those vlogs are indeed interesting and helpful.

In quantitative data analysis, the selected paper only provided basic descriptive statistics, which failed to provide enough insights for the audience. In Section 5.1.2, it is reported that students harbour an overall positive attitude and teachers also show a ‘slight agreement’ on using vlogs. The reporting style devalues the significant difference between learners and practitioners in this aspect, as demonstrated by the effect sizes I reconstructed based on the descriptive statistics (Hedges’ g = 1.176, 95% Cohen’s d = .79 to 1.56; Cohen, 1988). Concerning qualitative data analysis, one vital issue is that the results sometimes deviate from what the study should have been focusing on. When demonstrating how vlogs contributed to rectifying pronunciation (S9, p. 11), the statement does not emphasise the unique role of vlogs but the benefit of captions which also exist in multiple other types of videos. Also, the participants only talked about extensive viewing on page 12, but they only acknowledged the usefulness without commenting on whether the programme encouraged them to do so.

🕵 In conclusion, although the selected paper touched on a relatively ignored aspect of L2 learning, it can be significantly improved concerning both its research design and interpretation. The present essay, firstly, has attempted to show how more attention could be paid to the speciality of vlogs in their genre, register, and style. Then, I reflected on the issues about research design (e.g., lack of control group and replicability), instrumentalisation (e.g., the validity of the questionnaire), and the interpretation of results (e.g., the misunderstanding of statistical findings).

📃 Materials mentioned in this post

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.).

Routledge. Dörnyei, Z., & Taguchi, T. (2009). Questionnaires in second language research: Construction, administration, and processing. Routledge.

Halliday, M. A. K., & Matthiessen, C. (2014). An introduction to functional grammar. Routledge.