Research & Discovery

Say Goodbye to the Memory Wall

In 1994, University of Virginia computer science professor emeritus William Wulf and his then-graduate student, Sally McKee, identified what would become a defining challenge in the field of computer science for decades to come. They called it the “memory wall.”

The memory wall results from two issues: outdated computing architecture, with a physical separation between computer processors and memory; and the fact that a processor can run much faster than the speed at which memory chips can provide data.

As early as the 1980s, researchers were predicting that computer systems could not keep up with the future trajectory of data. Then came the internet of things – devices connected through the cloud collecting vast amounts of data. The rapid growth of bioinformatics has been another source of the data explosion.

By 2018, Forbes reported that 90% of the world’s data had been generated in just the previous two years. The servers processing these data have not been able to keep up and provide timely results, such as identifying new COVID variants or responding quickly when a patient falls ill.

That was the year when researchers in the University of Virginia’s Department of Computer Science and Charles L. Brown Department of Electrical and Computer Engineering were selected to establish a $29.7 million research effort to remove the memory wall.

Four years into the five-year grant, the UVA-led, nine-university Center for Research in Intelligent Storage and Processing in Memory, or CRISP, has made strides that match the gargantuan problem the center is trying to solve.

The center’s investigators and graduate students have published 378 papers, released 26 new software tools, and submitted 18 patent applications, of which two have been granted by the U.S. Patent and Trademark Office.

“We are four years into developing novel architectures that will benefit society in ways not even imaginable a few years ago,” said Kevin Skadron, Harry Douglas Forsyth Professor of Computer Science at the UVA School of Engineering and Applied Science and center leader.

The new architectures the CRISP collaborators are developing integrate processing and memory into a single unit. By tightly coupling the processing into the data storage, the processing rate can be dramatically increased.

The fundamental redesign is overdue and equates to past evolutions in computing, like the introduction of integrated circuits and the paradigm shift from mainframes to personal computers and workstations.

Battling Cancer and COVID

One of the center’s early wins came in the fight against cancer.

The key to targeted cancer treatments is analyzing DNA samples to find patterns in genetic data, which then pinpoint specific treatments based on epidemiology. Center researchers set out to see just how much they could speed up that process, which scientists call “sequence alignment.”

The results were stunning. Their new architectures could shorten sequence alignment time from 20 hours to less than a second. Center researchers also projected they could speed this up 100 times further in future evolutions of their processing redesigns.

“This one example highlights the importance of our collaboration with other universities, across multiple disciplines, to remove the memory wall,” Skadron said following the center’s second-annual review in November 2019. “Industry and government are working with us to realize the incredible breakthroughs that can occur with large data sets. All sectors of our economy and society will benefit.”

Engineering professor Kevin Skadron leads the nine-university center that is creating the ultra-fast computing systems that pair data processing and memory in a single unit. (Photo by Tom Cogill)

As center researchers headed back to their labs to build on these exciting results, the first cases of a novel coronavirus were appearing in Wuhan, China.

By the spring of 2020, the world was in lockdown from the COVID-19 pandemic. So the center’s researchers added another real-world case study midway through the grant cycle. They joined the global scientific community in efforts to tackle SARS-CoV-2.

Effective mitigations would require accelerated pathways to understanding the virus’ methods of transmission and mutation. Large numbers of biological samples from humans infected by the virus were being collected from wastewater, and these could be used to sequence the virus to get at this information.

But processing just one sample would take weeks with today’s computers. Faster results were needed to get ahead of the virus’ spread and inform tactics for halting it. This is exactly where the center researchers’ hard work would prove invaluable.

UVA Engineering’s Samira Khan, left, and Mircea Stan are among the UVA researchers guiding center efforts. (Contributed photos)

To get at the viral sequences, they could apply the ultra-fast processing methods they created for targeted cancer treatments. They could also draw on their research for new computing methods that eliminated other data bottlenecks in the coronavirus genomics pipeline. The dramatic results sped up the processing timeline so epidemiologists could get actionable insights from samples within a few hours.

Researchers could even backtrack the sequences fast enough to identify transmission networks in micro-detail, thanks to the new processing methods, providing a powerful example of just how critical these next-generation computing architectures are for society.

Ultra-fast computing will become a key player in the defense against new diseases that emerge with no historical context. Being able to sort through new streams of biomedical information, like the CRISP researchers did to get at methods to predict COVID-19’s next moves in real time, will be the only way to track disease outbreaks and develop methods of control.

These same methods are the key to better medical treatments for a myriad of existing diseases, too, in addition to cancer. The researchers have continued their work throughout the pandemic conducting acceleration studies of new hardware and software.

What’s Next

The UVA-led center has funded 185 graduate students across the participating universities, 59 of whom have graduated and gone on to jobs in important sectors such as the U.S. semiconductor industry and as faculty in U.S. universities. Skadron said the center’s work has also provided opportunities for undergraduate researchers at UVA and supported innovations in curriculum for computer systems design.

The center is part of the Joint University Microelectronics Program funded and managed by North Carolina-based Semiconductor Research Corporation, a consortium that includes engineers and scientists from technology companies, universities and government agencies.

UVA’s team includes Skadron; Samira Khan, assistant professor of computer science and an expert in computer architecture and its implications for software systems; and Mircea Stan, Virginia Microelectronics Consortium Professor in electrical and computer engineering and an expert in the design of high-performance, low-power chips and circuits.

CRISP investigators gathered at UVA in 2019 for their second annual research review. A few months later, they would join global scientists in the race to tackle COVID-19. (Contributed photo)

Center collaborators are Cornell University; Georgia Tech; Pennsylvania State University; the University of California, Los Angeles; the University of California, San Diego; the University of Washington; the University of Wisconsin; and the University of Pennsylvania.

In the final year of the grant, the center’s investigative teams will continue testing their new architectures in three primary areas of application: targeted cancer treatments, analytics for large datasets and video analysis.

By the end of 2022, they plan to demonstrate methods to identify a targeted cancer therapy in 24 hours, execute big data processing that is 100 times faster than state-of-the art, and power artificial intelligence that can scan videos in real time to accurately label objects and identify specific movements.

“This giant leap in computing architectures will benefit other human endeavors even beyond medicine, such as smart cities and autonomous transportation,” Skadron said. “We are honored for the opportunity to contribute to society in such a profound way.”