What We Know About Pandemic Impact on Learning

A sweeping review of national test data suggests pandemic impact on student learning may not be as bad as we feared. According to this study, the pandemic-driven jump to online learning has had little impact on children’s reading growth and has only somewhat slowed gains in math. That positive news comes from the testing nonprofit NWEA and covers nearly 4.4 million U.S. students in grades three through eight. But the report also includes a worrying caveat: Many of the nation’s most vulnerable students are missing from the data.

“Preliminary fall data suggests that, on average, students are faring better than we had feared,” says Beth Tarasawa, head of research at NWEA, in a news release accompanying the report. “While there’s some good news here, we want to stress that not all students are represented in the data, especially from our most marginalized communities.”

Until now, estimates of learning loss have been just that — estimates or projections, based on the kind of academic backsliding schools see after a long summer. This report offers the clearest picture yet of the impact that the past eight months of disruption have had on student learning.

The MAP Growth Test

The data at the heart of NWEA’s report come from what’s known to teachers and children alike as the MAP Growth test — a check-in assessment used to measure kids’ math and reading skills that’s generally given three times a year, in fall, winter and spring. While millions of students took these MAP tests in the winter of 2020, few took them again in the spring as schools raced (and many struggled) to provide learning online. But this fall, nearly 4.4 million children did take the test, either from home or back in a classroom. And the results give researchers a vital new data point: a measure of where students are right now.

Tarasawa and her research team studied the data a few different ways. First, they compared students’ performance this fall — in, say, third-grade reading — with the performance of a different group of students who took third-grade reading in the fall of 2019, before the coronavirus pandemic. Tarasawa tells NPR that with this method of comparison, the results in reading were “relatively optimistic” because “kids on average are performing similarly to how [other children] did pre-pandemic.” In math, the current pandemic class of students performed about 5 to 10 percentile points lower than the pre-pandemic comparison group — what Tarasawa describes as a “moderate” drop.

In addition to comparing two different groups of students, researchers also measured pandemic impact by studying students’ individual growth over time, looking at where they were when they took the MAP test in the winter of 2020 and comparing it with where they are now, in the fall of 2020.

“We saw, on average, students showed growth in both math and reading across the grade levels in almost all grades,” says Tarasawa. “Most students made some learning gains in both reading and math since COVID started.” In short, students kept learning when schools shifted online; they just didn’t learn quite as much in math as they likely would have if there had never been a pandemic.

Mitigating the learning loss that is happening will still require patience and a thoughtful approach, says Aaliyah Samuel, NWEA’s executive vice president of government affairs and partnerships. “Addressing the unfinished learning is going to be a matter of time. We really need to be thinking about the supports and interventions for kids over at least a two- or three-year runway,” she says. Depending on the depth of learning lost, school districts could consider a range of options, including extending the school year or even enlisting a volunteer tutoring corps.

Roughly a Quarter of Students Missing

The “good” news (and the not-so-good news) in this report also comes with an important and worrying red flag. In an effort to be sure their 4.4 million-student sample, albeit large, was also representative of America’s classrooms, NWEA researchers dug into the demographics of this new data set and compared it with the earlier fall 2019 test data — a sample of nearly 5.2 million children.

What they found, Tarasawa says, is that roughly a quarter of students were missing — meaning they didn’t take the MAP test this fall — and that these children are “more likely to be black and brown, more likely to be from high-poverty schools and more likely to have lower performance in the first place.” The researchers cite a host of possible reasons these students weren’t able to take the latest test, including a lack of technology or Internet access at home, as well as the possibility that some children have disengaged from school more broadly. “This is screaming that we have to be very cautious,” says Tarasawa, about interpreting the relatively optimistic results in reading and even math as evidence that the kids are all right.

“It’s just like any time you get a new puzzle,” Samuel says. “The first thing you do is … you start to look for the corners because those are usually the easiest to put together first.” That’s where we’re at now, she explains: building the edges of the puzzle. Nevertheless, as far as pandemic impact on student learning, it seems pretty clear which communities are struggling the most.

Allison Green
Boston Tutoring Services