COVID-19 and the Learning Loss Dilemma

Although statistics vary across provinces, Canadian schools in general were closed for a total of 51 weeks during the pandemic – placing the nation in the highest bracket globally for school closures (UNESCO, n.d.). Unsurprisingly, provincial policymakers across Canada continue to be concerned about the negative short- and long-term impacts of the disruptions created by these closures on students’ learning and have been focusing their attention on improving achievement in traditional content areas such as reading, writing, mathematics, and science. The dominant political and popular media discourse is that students have fallen behind and need to “catch up” in these foundational subject areas. Certainly, international research suggests that this concern is well-founded, and that students’ learning has been significantly disrupted during the pandemic.

Learning loss within and outside of Canada

International research is beginning to document the learning losses students experienced due to school closures, shifts toward online and hybrid learning, and other impacts associated with successive waves of the pandemic. Although these studies are relatively sparse, a limited number of Western nations such as the Netherlands (Engzell et al., 2021), Germany (Depping et al., 2021), Belgium (Maldonato & De Witte, 2021), and the U.S. (Bailey et al., 2021) suggest learning essentially stalled during the pandemic. These studies also suggest that the pandemic may have exacerbated existing inequalities, with lower socio-economic status (SES) students falling even further behind their more affluent peers. Collectively, the emergent literature suggests that learning and the academic resilience of students globally have been particularly threatened during the pandemic (Volante & Klinger, 2022a).

Unfortunately, Canadian large-scale assessment research, which is used to draw reliable and comparative measures of student achievement and system-level judgments, has been particularly constrained during the pandemic. Indeed, the administration of international, national, and provincial assessments have all been adversely impacted, with numerous cancellations during the initial waves of the pandemic. Further, those assessment programs that did occur met with high levels of non-participation, impacting sampling designs. These challenges have made it difficult to make provincial comparisons of student achievement. Those studies that do exist are confined to select geographical contexts such as Toronto (Toronto District School Board, 2021), or offer predicted losses extrapolated from summer learning research (Aurini & Davies, 2021). Collectively, the available research in Canada has been unable to quantify, with any level of certainty, the pandemic’s impact on students’ achievement.

Nevertheless, Canadian education systems, including higher education settings, are reporting important gaps in student learning, suggesting that learning losses have occurred for students in K–12 and beyond. International organizations such as the Organisation for Economic Co-operation and Development (2020) and the United Nations Educational, Scientific and Cultural Organization (2022) have also reported that lower SES students and their families have been unable to secure the necessary resources needed to succeed in online and hybrid learning environments amidst the turmoil created by the pandemic. These challenges are also well documented in popular media stories across Canada and reflected in the policy interventions adopted by various provincial governments to try to support our most vulnerable student groups. Nonetheless, the relative success of these efforts and interventions has not been measured.

Policy trends across Canada

One of our recent studies provided a pan-Canadian analysis of educational policy developments from January 2020 to December 2021 that were specifically related to academic resilience in the wake of the initial waves of the pandemic. Not surprisingly, our findings suggested greater attention was devoted to academic issues – namely learning outcomes in cognitive domains – with relatively fewer policies and resources to support mental health and general physical wellness (Volante et al., 2022c). Our analysis also suggested that there was also a general lack of policy differentiation in terms of how specific resources and supports were to be directed within provincial educational jurisdictions to help support at-risk students. Without such differentiation, we have argued that the resources developed will not be fully realized, and will undoubtedly fail to stem the growing disparities between low- and high-SES student populations that have been amplified by the pandemic.

Collectively, our policy research also underscores the importance of reconsidering how provincial education systems operate to achieve positive outcomes for students and how these outcomes might be “measured” and evaluated. Although a great deal of work is already underway by provincial testing bodies, large-scale assessment measures currently do not offer a multifaceted picture of student development. Conversely, international achievement measures such as those administered by the OECD and/or the International Association for the Evaluation of Educational Achievement (IEA), provide background questionnaires that attempt to capture student-, school-, and system-level factors that may be related to student outcomes. As an example, these international measures increasingly include factors and outcomes that could be classified as non-cognitive skills, drawing attention to the importance of non-cognitive and mental/physical wellness outcomes.

Challenging the dominant discourse

One would be hard-pressed to find any educational stakeholder group that does not recognize the importance of student achievement in traditional subject areas such as reading, writing, mathematics, and science. Nevertheless, the pandemic has highlighted how achievement in traditional cognitive domains offers a necessary, but incomplete picture of the pressing challenges that face Canadian youth. As Volante, Klinger, and Barrett (2021) noted in a previous Education Canada article, Canadian children reported disturbing trends in relation to mental health and general wellness. Similarly, the promotion of non-cognitive skills such as growth mindset represents an increasingly important cadre of key attributes that contribute to resilient students, schools, and education systems in general (Volante & Klinger, 2022b).

Thus, provincial policymakers are faced with an important dilemma. Namely, to develop a comprehensive vision of student learning and wellbeing that emphasizes cognitive (i.e. reading, writing, mathematics, science achievement), non-cognitive (i.e. learning habits, self-beliefs, growth mindset), and general wellness in the face of dominant historical and political ideologies that have focused almost exclusively on standards-based education reform. Indeed, standards-based reform and achievement of the “three R’s” (reading, writing, arithmetic), has largely driven large-scale reform agendas in much of the Western world for more than half a century (Volante et al., 2022d). In spite of the concerns and evidence that have arisen with respect to the impact of the pandemic, every provincial jurisdiction in Canada continues to adhere to a standards-based reform model that emphasizes a hierarchy of subject areas and achievement outcomes. The importance of other critical factors and outcomes may be acknowledged, but receives little if any formal attention, and there is little effort to build on the information being collected by international assessments that now include such measures.

Rethinking large-scale reform

It is often written that adversity is a catalyst for growth and change. Certainly, the last several years have likely presented the most formidable adversity that many students, families, and teachers may face in their lifetimes. Rather than return to status quo approaches that emphasize a narrow set of achievement outcomes, this critical epoch in our collective history offers an opportunity to rethink our approaches to large-scale education reform to provide a more nuanced recognition of the skills and attributes required to face the challenges of the future. Certainly, any student, parent, or teacher will tell you that more than academic content was lost during the pandemic – capturing and addressing the multifaceted complexity of this “loss” requires a new conception of what quality education looks like in a post-COVID world. Failing to recognize the latter could undoubtedly result in students catching up in academic content, only to fall behind in the non-cognitive skills they require for further success. It is time for us to look for ways to link provincial, national, and international assessments and surveys in order to obtain the data needed to examine the complexity of learning that supports the whole child.

This research is supported by the Social Sciences and Humanities Research Council of Canada (SSHRC).

References

Aurini, J., & Davies, S. (2020). COVID-19 school closures and educational achievement gaps in Canada: Lessons from Ontario summer learning research. Canadian Review of Sociology, 58(2), 165–185. doi.org/10.1111/cars.12334

Bailey, D. H., Duncan, G. J., Murnane, R. J., & Yeung, N. A. (2021). Achievement gaps in the wake of COVID-19. Educational Researcher, 50(5), 266–275. doi.org/10.3102%2F0013189X211011237

Depping, D., Lücken, M. et al. (2021). KompetenzständeHamburger Schülerinnen vor und während der Corona-Pandemie [Alternative pupils’ competence measurement in Hamburg during the Corona pandemic]. DDS – Die Deutsche Schule, Beiheft, 17, 51–79. www.pedocs.de/volltexte/2021/21514/pdf/DDS_Beiheft_17_2021_Depping_et_al_Kompetenzstaende_Hamburger.pdf

Engzell, P., Frey, A., & Verhagen, M. D. (2021). Learning loss due to school closures during the COVID-19 pandemic. Proceedings of the National Academy of Sciences of the United States of America, 118(17), 1-7.www.pnas.org/content/pnas/118/17/e2022376118.full.pdf

Maldonato, J. E., & De Witte, C. (2021). The effect of school closures on standardised student test outcomes. British Educational Research Journal. doi.org/10.1002/berj.3754

Organisation for Economic Co-operation and Development (2020). The impact of COVID-19 on student equity and inclusion: supporting vulnerable students during school closures and school re-openings. OECD Publishing. https://oecd.org/education/strength-through-diversity/OECD%20COVID-19%20Brief%20Vulnerable%20Students.pdf

UNESCO. (n.d.). Dashboards on the Global Monitoring of School Closures Caused by the COVID-19 Pandemic.  https://covid19.uis.unesco.org/global-monitoring-school-closures-covid19

Volante, L., & Klinger, D. A. (2022a). PISA, global reference societies, and policy borrowing: The promises and pitfalls of academic resilience. Policy Futures in Education. https://journals.sagepub.com/doi/10.1177/14782103211069002

Volante, L., & Klinger, D. A. (2022b, January 27–28). Assessing non-cognitive skills to promote equity and academic resilience [Paper presentation]. Advancing Assessment and Evaluation Virtual Conference. https://aaec2022.netlify.app/_main.pdf

Volante, L., Klinger, D. A., & Barrett, J. (2021). Academic resilience in a post-COVID world: a multi-level approach to capacity building. Education Canada, 61(3), 32–34.

Volante, L., Lara, C., Klinger, D. A., & Siegel, M. (2022c). Academic resilience during the COVID-19 pandemic: a triarchic analysis of education policy developments across Canada. Canadian Journal of Education, 45(4), 1112–1140.

Volante, L., Schnepf, S., & Klinger, D. A. (Eds.) (2022d). Cross-national achievement surveys for monitoring educational outcomes: Policies, practices, and political reforms within the European Union. Publications Office of the European Union. https://data.europa.eu/doi/10.2760/406165

Photo: iStock
First published in Education Canada, April 2023

Meet the Expert(s)

Dr. Louis Volante

Professor, Brock University & UNU-MERIT

Louis Volante, PhD, is a Professor at Brock University and a Professorial Fellow at UNU-MERIT/Maastricht Graduate School of Governance. His current research is focused on multi-level education governance, comparative policy analysis, impact evaluation of policies and programs, politics of education reform, international large-scale assessments and transnational governance, and cross-national educational inequalities.

Read More

Dr. Don A. Klinger

Pro Vice-Chancellor, Division of Education, University of Waikato

Don Klinger, PhD, is Pro Vice-Chancellor for Te Wānanga Toi Tangata Division of Education at the University of Waikato. His research focuses heavily on the formative and summative methods used to measure student outcomes, and the resulting decisions and actions made with assessment data.

Read More