Study: Tutoring May Mitigate Pandemic Learning Losses

The latest federal stimulus package included funding for K-12 school districts across the country to use specifically for combatting learning losses resulting from the pandemic.

These funds have been earmarked for acceleration programs that education experts believe can help make up some of the ground.

A few of the more talked-about programs involve “high-dosage tutoring” and “vacation academies.”

Recently, University of Virginia education assistant professor Beth Schueler co-wrote a policy brief with Carly D. Robinson, Matthew A. Kraft and Susanna Loeb at Brown University’s Annenberg Institute that discussed how both can be viable methods and what the research says about how to design effective programs. The brief was written for policymakers and practitioners currently crafting their approaches to helping students who have fallen behind due to COVID-19.

“I think tutoring and small-group instruction is particularly well-suited to this moment, and not just because of effects on student academic outcomes,” Schueler said. “Don’t get me wrong, learning is important and the impacts of these programs on academic achievement are impressive, but these are also opportunities to build and rebuild relationships with people in a time when I think we’ve all been disconnected from each other and starved for those relationships.

“Kids can be struggling on a number of dimensions at home, and so taking the time to provide some individualized attention could be really valuable from the perspective of not only academics, but also broader student wellbeing.

“And from a teacher’s perspective … some may really be eager to reconnect with students and earn extra income, in a direct way, where they’re not having to manage a huge class.”

Schueler says evidence suggests that having the same tutor over a period of time is beneficial. (Contributed photo)

UVA Today caught up with Schueler to learn more.

Q. Can you tell us what “high-dosage tutoring” and “vacation academies” mean?

A. Both refer to intensity – either in terms of dosage of tutoring multiple times per week, or in the case of vacation academies, weeklong programs that are run over vacation breaks focused on a single subject for the duration of the program.

We focused in this brief on a variety of ways to individualize instruction, with high-dosage tutoring being one of them. The research suggests tutoring is most effective when it’s delivered in three or more sessions a week that each last for 30 to 60 minutes per day for at least 10 weeks. Tutoring can be effective when delivered to between one and four students at a time. The weeklong vacation academy programs are somewhat larger, with 10 students per group, and led by a teacher selected based on talent.

Q. Was your research based on virtual, in-person tutoring or a combination?

A. Most of the research so far has focused on in-person tutoring. There has been some preliminary evidence that suggests promising results for virtual tutoring, but whether the large gains can be replicated virtually remains an open question researchers are trying to explore in a lot more detail right now given that many communities are continuing to practice physical distancing due to COVID-19. Virtual programs also might represent a more affordable and scalable model than in-person tutoring.

Q. In your brief, you say research has provided strong evidence that high-dosage tutoring can produce large learning gains for a wide range of students, including those who have fallen behind academically. Can you tell our readers some of the specifics there in terms of those gains, which grade levels showed the most improvement and how subject matter – reading or math – played into it?

A. The effects that you see from some of these programs are quite large relative to other educational interventions that have been studied. For most educational programs, you’ll have observational data that researchers worry doesn’t isolate the effect of the program itself from all the other factors contributing to student outcomes, but with tutoring that’s just not the case.

We have a very persuasive literature where there have been opportunities to randomize students to receive individualized instruction, which allows researchers to credibly identify the causal effect of that program specifically. And there have now been a number of what are called meta-analyses, where researchers look across a large literature and review all rigorous studies of those interventions to estimate the average effect across those programs. There’s been a few of those in recent years that have shown consistently large positive effects.

And that’s been true in both reading and math, which is exciting because in some case, for example, in educational accountability policy interventions you’ll often see math scores move but not reading scores. In this case, the research has documented successful programs in both subjects.

It’s also been across a range of ages and grade levels. In recent years, there’s been some really interesting and compelling evidence on tutoring for high school students and also small-group instruction for middle and high school students that suggests that these kind of state-of-the-art, well-designed programs can have a big impact even at the high school level.

You’ll hear observers make the argument that it’s too late to intervene or money would be spent better earlier in a child’s developmental trajectory, and so the fact that tutoring also seems effective for these later grade levels is exciting.

Q. You acknowledge in the paper that tutoring can be expensive, but that its effects make them cost-effective relative to many other educational interventions. Can you explain that a little more?

A. There’s a particular study where the researcher did a cost-benefit analysis of a bunch of different educational interventions, and tutoring outperformed most of the other interventions. These programs are expensive and have high up-front costs, but you’re getting such a big bang for your buck that even though it requires a significant investment, the benefit is worth the cost in the end.

Despite the fact that there’s persuasive research showing these programs are cost beneficial, I still think the high upfront price tag deters some schools and districts and states from adopting these programs on a wide scale. A lesser-known approach to individualizing instruction is vacation academies, which are basically small-group programs held over vacation breaks. Instead of a 1-to-2 or a 1-to-1 teacher-student ratio, these are roughly 1-to-10. For these programs, the districts hire teachers who they consider to be talented and have them work with a small group of students who are struggling in a particular subject and they work on that single subject over weeklong vacation breaks.

I’ve studied these programs in a couple of low-performing school districts now and found that they also produce impressive results – not quite as large as the size of the effects that you’d see with a state-of-the-art, high-dosage tutoring program, but that’s not super surprising given the fact that they are weeklong programs as opposed to yearlong programs and are less expensive than tutoring.

So there are other models that capture the spirit of what high-dosage tutoring is trying to do, which is to tailor and individualize instruction for particular student needs, something that is difficult to do when you have a class of 30 kids, but much easier with a smaller group. There are creative ways to do that with small-group programs that have typically been implemented over vacation breaks during the school year, but that you could imagine being adapted into longer programs over summer break.

Q. Does it make a difference if a child, say one who is elementary school, always has the same tutor?

A. There is some evidence that consistency is good. I would encourage people to read the brief, because I worry that districts are getting the message now that “We’ve got to do tutoring” and that many may rush to do that without thinking carefully about the details. There’s a danger in doing so without attention to quality and to designing a program that’s going to have an impact.

This consistency of the tutor, the teacher, is part of that. You can imagine, for example, a volunteer-based program where you have a rotating cast of characters coming in and out of the kid’s life that might not produce the same gains as a program where the tutor gets to know the student and their academic and social needs over a full semester or year.

With the vacation academies, one finding was that students who stayed with the same teacher for the whole program had declines in their exposure to exclusionary discipline. It reduced their suspensions [from school] after the program, which suggests something about the importance of relationship-building.

Q. What was the biggest takeaway from the research you did on the vacation academies?

A. It would probably be that the folks who designed those programs put a lot of effort into framing and pitching the [concept] to students and families as a positive and fun opportunity, rather than a soul-crushing, test-prep-oriented experience. They’ve done a lot to make it a joyful environment for kids and teachers.

I imagine that’s going to be key as we address the educational consequences of the pandemic – to not shy away from academics entirely during this time because we’re so worried that children are overwhelmed, but to recognize that a lot of kids probably are overwhelmed and to create supportive and enjoyable experiences to make sure that they show up, participate, and get re-engaged with school.

Q. With more and more kids returning to schools, probably even more as we approach the fall, are there any things in particular pertaining to this research that you’ll be keeping a close eye on?

A. With some collaborators here at UVA, we are partnering with the state of Virginia to look at COVID reopening plans across divisions and how equity of student outcomes has changed over time, pre- and post-pandemic. I will be interested to see the extent to which divisions are providing tutoring and individualized instructional opportunities and whether that seems to play a role in producing positive outcomes.

That’s true not only for the Virginia context but also nationally. How will districts and states take from this impressive literature to compensate for the lost learning time that occurred because of the pandemic? I think that will be a big area to keep an eye on.