U.Va. Study: Turnaround Schools in California See Striking Early Success

May 8, 2012 — Among California's persistently lowest-achieving schools, those that have implemented aggressive turnaround reforms mandated by the federal government are showing significant improvements just one year later, a new study finds.

The study provides early evidence that the Obama administration's $3.5 billion investment in improving the nation's most chronically underperforming schools is working, according to study author Thomas S. Dee, a professor of public policy and economics at the University of Virginia's Frank Batten School of Leadership and Public Policy.

The study, "School Turnarounds: Evidence from the 2009 Stimulus," was recently published as a working paper by the National Bureau of Economic Research, where Dee is a research associate.

One year after 82 of California's worst-performing schools enacted major reforms as a condition of receiving federal funds from the School Improvement Grant program – worth up to $2 million per school annually over three years – those schools, on average, have closed 23 percent of their achievement gap in meeting the state's performance targets for student test scores, the study found.

These results are surprising and strikingly significant after just one year, Dee said. "I was surprised that there were positive effects at all within the first year," he said. "My presumption had been that schools sought out this money largely because they were in fiscal crisis. I worried that the local buy-in to the required reforms would be poor and that the implementation would be uneven. Instead, what I found were sizable first-year improvements in school performance as a result of the SIG-funded reforms.

"These results provide encouraging, early evidence that aggressive, multi-faceted, well-funded reforms can catalyze meaningful improvements in the schools serving our most vulnerable children," said Dee, who, in addition to his primary appointment in the Batten School, has a joint appointment as a professor of economics in the College of Arts & Sciences and a research professor of education in the Curry School of Education. He is also a member of U.Va.'s Center on Education Policy and Workforce Competitiveness.

U.S. Secretary of Education Arne Duncan revamped the longstanding School Improvement Grants program to focus on what he called "transformation, not tinkering" at the nation's "persistently lowest achieving" public schools – those performing in the bottom 5 percent of all schools in a state, Dee's study explains. Beginning in 2010, as a condition of receiving School Improvement Grant funds, those schools had to enact one of three federally prescribed, multifaceted reform models for achieving school turnarounds.

The three reform models each require a package of reforms that emphasize, in varying combinations, leadership change, data-driven and differentiated instruction, and performance-based teacher evaluations, as well as "wraparound" community services like health and nutrition programs designed to support students' readiness to learn.

Dee's study focused on California schools, he said, because they received far more of the revamped grants than any other state, claiming a tenth of all the turnaround grants for the 2010-11 school year, worth $1.5 million per school on average.

The study compares "just eligible" schools – where performance was just low enough to place them in the lowest 5 percent – to schools that were "just ineligible," with performance slightly above the 5-percent threshold. These schools on either side of the 5 percent threshold are essentially indistinguishable in terms of their baseline traits, Dee said.

Compared with the "just ineligible" control group, the School Improvement Grant-funded schools, on average, saw dramatic improvements in their subsequent 2010-11 test scores, closing 23 percent of their achievement gap with respect to the state performance target.

On average, the grant-recipient schools hired more teachers, resulting in roughly five fewer students per teacher. With the mix of teacher turnover and new hiring at these schools, the average level of teacher experience fell by two years.

These reforms are expensive in absolute terms – roughly $1,500 per pupil – but "fairly cost-effective," Dee said, comparing favorably with the cost-effectiveness of class-size reductions as quantified in other studies.

"Whether these promising improvements have been replicated in other states and can be sustained is still an open question, one that my research team and I will be studying carefully," Dee said.

Duncan has already taken notice of the study. "These data are still preliminary," he said in a May 1 press release. "Several years of data will be needed to demonstrate robust, long-term growth in student outcomes in SIG schools. But Dee's careful study belies the conventional wisdom that little can be done to significantly boost student achievement in low-performing schools.

"Dee's new study reminds us that poverty is not destiny."

– by Brevy Cannon