A University of Virginia study examining the impact intensive college advising has on college graduation rates of low-income, first-generation students is one of three winners of a national competition held in response to the White House call for evidence-based reforms. 

Ben Castleman, an assistant professor of education and public policy at U.Va.’s Curry School of Education, is examining the effectiveness of “Bottom Line,” a nonprofit organization that provides one-on-one advising to low-income, first-generation students beginning in their senior year of high school and, in most cases, continuing throughout their undergraduate careers.

Of the competition’s three winners, the Bottom Line evaluation was the only one in the education arena. 

The competition, launched by the Coalition for Evidence-Based Policy, “demonstrates the feasibility and value of low-cost randomized controlled trials for building important evidence about what works in social spending,” said John Baron, the coalition’s president.

Such trials “are common in medicine to evaluate the impact of new drug treatments,” Castleman said. “They are less common in education, but just as important to isolate the impact that educational interventions have on students’ attainment and achievement.”

Bottom Line was founded in Boston in 1997 and is currently operating in Massachusetts, New York and Illinois. The organization features two primary programs: College Access and College Success. 

In the Access Program, full-time counselors meet one-on-one with high school seniors to guide their college search, identify target schools, complete applications for admission and financial aid, and finally make a decision about where to attend based on fit and affordability. Roughly two-thirds of students in the Access Program continue into the Success Program, while other students enter through an open online application or via community partnerships.

The College Success Program begins the summer after high school with a summer bridge program, preparing students for the rigors of college and helping to reduce “summer melt,” the phenomenon that occurs to students who graduate from high school with the intention of attending college in the fall, but then don’t matriculate. Once the semester begins, students will receive up to six years of on-campus, relationship-focused support to help resolve problems that threaten to derail their progress toward graduation.

Bottom Line focuses on student academic degree progress, employability, financial aid and life. By building strong one-on-one relationships and monitoring progress, Bottom Line greatly enhances its students’ chances of success in college and beyond, earlier studies found.

“The ultimate goal of the Bottom Line program is more than getting in to college; it is ensuring that students ultimately earn a degree,” Castleman said. “Prior studies of the program suggest that it has a substantial effect on where students enroll and persist in college, leading them to institutions that are of the same quality but considerably more affordable than institutions attended by students who were not selected for the Access Program.”

Castleman’s research, now under way, is designed to determine whether these promising findings extend to higher rates of degree attainment. The multi-site study will track nearly 1,400 students over seven years to determine the program’s short-term, medium-term and long-term impacts.

A new element to this study is that the program has recently implemented a lottery system to accept participants, instead of its previous system of first-come, first-served.  The lottery system contributes to the integrity of the randomized control trial.

“What makes the project particularly compelling from an evaluation point of view is that the costs of implementing a random controlled trial are very low,” Castleman said. “Just by changing application procedures so that students are selected by lottery instead of first-come, first-served, Bottom Line is able to obtain rigorous evidence of the program’s impact on students’ long-term college outcomes.”

The coalition competition winners also included selected random controlled trials of Durham Connects, a postnatal nurse home-visiting program, and of workplace health and safety inspections conducted by the federal Occupational Safety and Health Administration.

In addition to intensive college advising, Castleman also explores lower-cost ways to support low-income, first-generation college students. His other current research includes examining the use of personalized text messaging to prevent summer melt. Castleman is also studying whether personalized texts can be used to increase the share of low-income students that receive and maintain financial aid in college.

The results of these texting interventions, as well as a broader examination of summer melt, are detailed in the forthcoming book “Summer Melt: Supporting Low-Income Students Through the Transition to College,” co-written by Castleman and Lindsay Page of the University of Pittsburgh. The book, published by Harvard Education Press, will be available in October.