Exploring digital literacy-raising pedagogy in emergent online English education

Case of the Chinese classes during Covid-19-related lockdown

Let's start with some background

Covid-19 😷  ‘forced’ language learners to receive emergent online education, especially in contexts where governmental policies enforce stronger virus-containing measures. The target context of the present study, China, falls into this category with a zero-Covid goal for the entire mainland area. Foreign language learning is particularly affected because it involves the features of interactive communication. In this regard, it is argued that modules with physical constraints, realia, and laboratory-based instruments are more influenced, whereas memory-based subjects may not suffer too much from this transition (Pather et al., 2020). As language learning is beyond the memory-based accumulation of linguistic stimuli, it cannot be immune from the negative aspects of a remote method (Paradis, 2009). It motivates this study to investigate how practitioners can moderate the negativity of this emergent transition by actively incorporating the instruction of digital literacy (DL).

DL, as ‘an essential requirement for life in a digital age’ (Bawden, 2008, p. 30), was understood as ‘the ability to understand and use information in multiple formats from a wide range of sources when it is presented via computers’ (Gilster, 1997, p. 1). Based on all sort of definitions around DL, Nelson et al. (2011) identified three core elements of this variable including ‘the skills and knowledge to use’, ‘the ability to critically understand the knowledge’, and ‘the capacity to create’ using digital devices. Following this categorisation, the present study will explore whether the instruction of DL will facilitate the general language learning process within emergent online education.

Research Question:

1.     Will the students be more engaged, involved, and willing to communicate in the emergent (forced) online English learning if digital-literacy-raising instruction was provided?

2.     If so, will these benefits further contribute to the final achievement of language learning?

✍️ What did we do?

Due to a new wave of the Covid-19 outbreak, most tertiary schools in Sichuan province changed their instruction type from classroom teaching to online teaching as of September 6^th. The present study chose 2 English classes (N1 = 49; N2 = 50) supervised by the same instructor at Sichuan University Jinjiang College. All the students regard Chinese or its dialects as their first language, with English as their second language (or first foreign language if mandarin Chinese is their second dialect). Fortunately, one week before the intervention, a diagnostic assessment was conducted for these students who had just progressed from Y1 to Y2, based on which all the students were randomly allocated into the two classes. Therefore, it is confirmed that the two classes share similar language proficiency before intervention as presented in Table 1.

The two classes were supposed to follow the same materials with the ultimate purpose of taking a standard exam at the end of the semester. Without changing the syllabus and course materials, for Class 1, the intervention represents an additional 10-minute guidance and practice of some effective ways of using the digital platform Tecent Meeting. The sample guidance includes the use of shortcuts to switch between silent/active mode, effective ways to locate and use the chat box, and methods to annotate the screenshots 💻. Also, the instruction is different in each class focusing on specific goals. For instance, if listening 👂 is the main target, the instructor provides some tricks to take digital notes of audio information.

In terms of data collection, apart from the demographic information and the pre-test results of the diagnostic assessment, after three weeks of online teaching, both classes were asked to measure their overall enjoyment (based on Dewaele & MacIntyre, 2014), involvement (self-developed instrument), and willingness to communicate (based on Wei & Xu, 2022)in these online classes. A mock exam was conducted at the end of the intervention to check the students' progress.

To address RQ1, group comparison is required, therefore, mean-comparison methods (e.g., t-tests) will be used. To address RQ2, the present study will use regression-based methods (e.g., linear regression and mediating analysis). Detailed procedures (e.g., reliability, validity, and assumption checking) will be discussed in the following section.

🍎What we found!

Reliability and Validity

Before analysing the data, to ensure the usefulness of the instrument, reliability and validity were tested. The indices of Cronbach’s alpha for enjoyment, involvement, and willingness to communicate are .800, .841, and .855, respectively. Although some items may not equally contribute to the reliability (e.g., Involvement item 4 with an item-rest correlation of .557), they were retained in later analysis because (1) the negative effects are ignorable and (2) the overall internal consistency is satisfactory. Therefore, the three scales were averaged as the overall score, which will be input into the statistical models later.

Effects of Digital-Literacy-Raising Pedagogy

A series of independent sample t-tests were performed to analyse the role of digital literacy-raising pedagogy in emergent online learning. Table 2 presents the results of four t-tests. Unfortunately, none of the four comparisons can reach the significance level of 95%. However, it should be noted that the present study only covered 99 participants, which can result in insignificant inference statistics (Field, 2009). Especially when we analyse the effect sizes (Cohen’s d) in this situation, it can be seen that the difference in WTC between the two groups is practically significant (d = .264).

Apart from the mean comparison, to further explore the real influence of the intervention, raincloud plots were employed to observe the distinction between data distributions. Figure 1 displays the raincloud plot of the Enjoyment score, with green representing the Experiment group and orange representing the Control group. From this figure, we could conclude that, although the means between the two groups are not significantly different, the Experiment group has a more clustered distribution. It means that more students belong to the high-achieving half. To compare, although the Control group has a similar mean, more students belong to the low-achieving group. Interestingly, the Control group harbours more top students at the end of the intervention. It may be because the intervention is not masked, which galvanises some students in the Control group out of the sense of competition, which needs to be analysed in future.

Figure 1: Raincloud plot of Enjoyment

Figure 2 presents the raincloud plot of involvement. Unlike the affective response of enjoyment, this more behavioural factor shows a general preference of the Experiment group. It means that digital-literacy-raising pedagogy does help the students involve in the English classes.

Figure 2: Raincloud plot of Involvement

Figure 3 is the raincloud plot of willingness to communicate (WTC), where the mean of the Experiment group is observably higher than that of the Control group. The standard deviation is much lower (i.e., the distribution is much narrower) with a ‘peak’ around the centre of the data. It highlights that the intervention successfully encouraged the students to be more willing to interact with the other participants.

Figure 3: Raincloud plot of WTC

The abovementioned three figures all show some signs of success of the intervention. Also, if comparing these three results, we can better understand the underlying mechanism of how digital literacy can contribute to the learners’ personal capital. Engagement, as an affective response, was promoted mainly for the average students who felt more joyful in online learning, whereas the low achievers in this aspect did not benefit too much. To compare, involvement as a behavioural dimension was increased generally with a smooth movement of the distribution curve. Willingness to communicate, as a cognitive style, enjoys a somewhat clustered enhancement with a peak appearing in the distribution.

The distribution of the scores in post-test, as presented in Figure 4, demonstrates a facilitation effect of the intervention on language achievement. Although the mean of the Experiment group is solely slightly higher than the Control group, most low-achieving individuals benefit significantly from the intervention. From the figure, the green part (Experiment group) has a clustered peak around the mean while the orange one (Control group) displays an even or normal distribution.

Figure 4: Raincloud plot of Post-test

In summary, the students are more engaged, involved, and willing to communicate in the emergent (forced) online English learning if digital literacy-raising instruction is provided. However, due to the low statistical power (i.e., small sample size), more replication studies are necessary to ascertain this finding.

📈 Multivariate Model

Section 3.2 addressed research question 1 by analysing the mean difference and data distribution of each variable. This section, to answer research question 2, will focus on a multivariate relationship among these factors. It is important because we only know whether but not how the final achievement can be affected by the selected factors.

Hierarchical regression was firstly used to test how the three factors can influence the post-test score together. As presented in Table 3 and 4, the pre-test score was input into the null model (Block 1) to control the effects of the basic language proficiency level (Adjusted R² = .368). Based on Cohen (1988), the control variable of pre-test exerts a moderate effect on the dependent variable (post-test), which also follows the researcher’s prediction. In the estimation model (Block 2), enjoyment, involvement, and WTC were entered to explore their overall influence. The R² Change of .132 indicates that these three selected variables altogether contribute to 13.2% of the variance of the dependent variable, which reached the ‘small’ benchmark proposed by Cohen (1988).

The coefficients summary of the hierarchical regression presented that, unlike involvement and WTC, enjoyment fails to exert statistically significant effects. This finding is not perfectly consistent with most previous studies investigating the role of enjoyment in language achievement (Dewaele & MacIntyre, 2014, 2016; Li et al., 2018). Nevertheless, considering the unique context of the current study, it may be because (1) affective reactions are more relevant in non-emergent pedagogy or (2) the enjoyment of online learning cannot override the perceived cumbersome of online learning. More studies are needed to help us better understand this relationship.

To complement the low statistical power, a machine learning model (boosting regression) was developed to successively enter information into a decision tree ensemble which in the end fit the data to the residual distributions (JASP Team, 2020). Table 5 and Figure 5 shows the summary statistics and relative influence of the entered variable on post-test.

Figure 5: Predictive Performance Plot and Relative Influence Plot

In conclusion, the benefits of digital literacy-raising pedagogy lie in its positive influence on the WTC of English learners who will actively involve in classroom interaction. This echoes one of the vital drawbacks of emergent online teaching, that is, the obliged utilisation of a teacher-centred pedagogy after years of pursuit of flipped classroom (cf. Lee & Wallace, 2018).

What does it mean?

Based on the findings of the present study, it is recommended that the instructors should provide appropriate training about how the students can maximise their efficiency in taking specific classes during emergent online learning. When designing the curriculum, teachers should reflect on the connection between learners’ learning strategies and the course aims. For instance, in the present study, the instructor recognised how the modifiability of learning materials through digital devices can streamline teaching. Hence, she introduced both the shortcuts of Tecent Meeting to annotate the screen and another note-taking software with the function of highlighting screenshots.

Also, beyond digital literacy, the practitioners can think about other ways to boost students’ WTC and involvement. For example, translanguaging can be a useful way to encourage domestic students to engage in class communication, although the practitioners should be cautious about the input quality of English when translanguaging is integrated.

In addition, for the online teaching in a more ‘normal’ context, the results of the present study can provide pedagogical insights as well. In future, if more studies confirm the role of digital literacy-raising efforts, this intervention could be incorporated by most foreign language teaching platforms.

Conclusion and Limitation

In conclusion, the present study has ascertained the importance of digital literacy in emergent online learning. First, descriptive statistics and mean comparison suggest that digital literacy-raising pedagogy may contribute to enjoyment, involvement, and WTC in the target online setting. Second, multivariate models confirm that involvement and WTC are the significant predictors of the final achievement. Pedagogical implications were also discussed.

Nevertheless, the present study also suffers from some limitations. First, due to the small sample size, the inferential statistics may not be accurate with low power. Second, although the internal consistency (reliability) was tested for all the instruments, the information about validity is insufficient. Third, the two achievement tests (pre-test and post-test) were not developed using robust criteria, which may result in the insensitivity of testing.

References

Bawden, D. (2008). Origins and Concepts of Digital Literacy. In Digital Literacies: Concepts, Policies, and Practices(pp. 17–32). Peter Lang Publishing.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Routledge.

Dewaele, J.-M., & MacIntyre, P. D. (2014). The two faces of Janus? Anxiety and enjoyment in the foreign language classroom. Studies in Second Language Learning and Teaching, 4(2).

Dewaele, J.-M., & MacIntyre, P. D. (2016). Foreign language enjoyment and foreign language classroom anxiety: The right and left feet of the language learner. Positive Psychology in SLA, 215, 236.

Field, A. P. (2009). Discovering statistics using SPSS: And sex, drugs and rock ‘n’ roll (3rd ed). SAGE Publications.

Gilster, P. (1997). Digital literacy. Wiley Computer Pub.

JASP Team. (2020). JASP (0.14.1).

Lee, G., & Wallace, A. (2018). Flipped Learning in the English as a Foreign Language Classroom: Outcomes and Perceptions. TESOL Quarterly, 52(1), 62–84. https://doi.org/10.1002/tesq.372

Li, C., Jiang, G., & Dewaele, J.-M. (2018). Understanding Chinese high school students’ Foreign Language Enjoyment: Validation of the Chinese version of the Foreign Language Enjoyment scale. System, 76, 183–196. https://doi.org/10.1016/j.system.2018.06.004

Nelson, K., Courier, M., & Joseph, G. W. (2011). An Investigation of Digital Literacy Needs of Students. Journal of Information Systems Education, 22(2), 95–110.

Paradis, M. (2009). Declarative and Procedural Determinants of Second Languages (Vol. 40). John Benjamins Publishing Company. https://doi.org/10.1075/sibil.40

Pather, N., Blyth, P., Chapman, J. A., Dayal, M. R., Flack, N. A., Fogg, Q. A., Green, R. A., Hulme, A. K., Johnson, I. P., & Meyer, A. J. (2020). Forced Disruption of Anatomy Education in Australia and New Zealand: An Acute Response to the Covid‐19 Pandemic. Anatomical Sciences Education, 13(3), 284–300.

Wei, X., & Xu, Q. (2022). Predictors of willingness to communicate in a second language (L2 WTC): Toward an integrated L2 WTC model from the socio‐psychological perspective. Foreign Language Annals, 55(1), 258–282. https://doi.org/10.1111/flan.12595

Appendix

Questionnaire

Demographics

1.     Age

2.     Gender

3.     Mother’s Highest Education

Enjoyment

1.     During the online learning of the past three weeks, did you feel happy when taking English classes?

2.     Did you become more willing to learn English online in future after these three weeks?

3.     Were you anxious when taking the recent online English courses?

4.     Compared to classroom learning, did you find online English classes interesting?

5.     Did you feel like you learn more effectively in an online setting?

Involvement

1.    In general, do you feel that online English classes in the past three weeks are more or less effective than classroom-based instruction in helping you improve your English skills?

2.    How often do you find yourself actively engaged in online English classes compared to when you are in a traditional classroom setting?

3.    How do you feel about the level of interaction you have with your classmates and instructors in online English classes?

4.    Do you feel like you are able to participate in online class discussions and activities as much as you would like to?

5.    Do you feel like you have enough opportunities to practice your English skills in an online class?

WTC

In the English classes over the past three weeks,

1.    I am willing to give more details about my opinions even without being asked.

2.    I take the initiative to ask questions in English in class.

3.    I volunteer my opinion in a whole class setting.

4.    I argue a point I feel is wrong with the classmates or teacher.

5.    I volunteer to share my personal opinions in small groups.

6.    I volunteer to participate in class debates.