Lost and Found: U.Va. Grad Student Discovers Ghosts in the MP3

Ryan Maguire, a Ph.D. student in composition and computer technologies in the University of Virginia’s McIntire Department of Music, has found that ghosts really do exist.

These audible ghosts are the lost sounds left behind during the creation of MP3 files.

Music and other sound content for iPods, computers and phones are sourced from audio files that are compressed for the convenience of storage and listening on a small device. In the compression process, a great deal of musical subtlety is lost. Maguire, who is affiliated with U.Va.’s Center for Computer Music, has made it his mission to expose and explain just what’s being taken away from the listening experience when audio is compressed into the MP3 format.

His project, “The Ghost in the MP3,” highlights the fragments of mysterious sounds left behind when the original audio version of Suzanne Vega’s late-’80s pop hit, “Tom’s Diner,” was reduced to a compressed nugget.

Vega’s hushed vocals contribute to the ghostly vibe, but Maguire didn’t choose the song for its atmosphere. Vega’s recording was used as a reference track during the development of the MP3 format. In tech history, “Tom’s Diner” is known as the “Mother of the MP3.”

Given the importance of Maguire’s research, it seemed like a good time to learn more about his project.

Q. MP3s emerged in the early ’90s, when storing and transferring large amounts of data was expensive. But nowadays, it still remains the standard music file – even though there are better alternatives and computers can handle larger and more complex files. What’s going on here?

A. MP3s have remained the standard because of the massive shift toward mobile computing with smartphones, tablets and laptops. While our computers have gained storage space, we’ve moved our digital lives over to smaller mobile devices, so that the storage available on an iPhone today is comparable to the storage space on a Mac Powerbook 20 years ago. So a file format that maximizes storage capacity while still sounding pretty good is still very useful.

Q. Why should we care so much about the sounds that are missing on compressed files?

A. There have been advances in audio engineering during the last two decades capable of producing amazingly detailed sounds that many people will never experience. Recording engineers often lament the fact that most listeners will never have the chance to truly hear what they have created in the studio, that their hard work will ultimately be filtered through inexpensive headphones via MP3 files.

Q. What exactly are these lost sounds?

A. They’re mostly sounds that the majority of people would have a difficult time hearing in their original context, due to a phenomenon known as auditory masking. Imagine two different instruments playing the same pitch at the same time, with one louder than the other. The quiet instrument’s sound will be partially covered up by the louder instrument. The sound still exists acoustically, but it’s very difficult to hear.

MP3s compress data based on this principle, assuming that in most contexts the sound won’t be missed by the average listener. It works very well for the most part, which is why we settle for it.

Q. Is your project more of an experimental avant-garde music/sound piece or research with an end product in mind?

A. My creative practice is by nature interdisciplinary, and so this is both art and research. There are two questions motivating my current work: “What are the sounds lost to MP3 compression?,” and “What kind of music could you create from those sounds?” And the answer to both of those questions is a musical answer – an answer in sound.

We’ve known for a long time what is altered during MP3 compression from a technical standpoint, but for a composer, and I think for the general public as well, it’s more interesting to hear it. And more interesting still is to create art from those sounds.

Q. What exactly is your mission in exposing what is being taken from the listening experience?

A. To create art, and in doing so, to inspire the same curiosity that I felt about this question in other people. The interdisciplinary practice of combining sound and idea that our Ph.D. program fosters is a vital way to make sense of the world, and it’s a privilege to share that with people.

Q. Where do you think your research will ultimately lead?

A. I’m currently working on more songs with this technique with the goal of releasing an album of this material in the near future. I’ve been thinking about experimenting with different ways of finding the “ghost” in real time, because the process is currently pretty time-consuming.

This all ultimately comes back around to my creative practice as a composer and sound artist. I am very interested in continuing to explore the boundary between what is perceptible and what isn’t – and using sound as a means to do so.

Q. What effects could your research have on the state of modern digital recording? That is, does your work have a practical application for future listening experiences?

A. I am essentially encouraging people to question why we accept MP3s as the de facto file format. If my research inspires people to listen closely to music with curiosity and interest, then that would be a tremendous accomplishment. By giving people an experience that they can connect with, we can move collectively toward higher-quality audio formats becoming standardized.

In the meantime, this work could provide audio engineers with a tool for better understanding how MP3 compression will ultimately affect their mixes. What’s more, there is an entirely unexplored sound world here that this research is uncovering, just waiting for artists to draw inspiration from.

Q. Have you experimented with this process on any other popular recordings?

A. Historically in the early ’90s, there was a series of listening tests administered in the development of the MP3 to scrutinize its efficacy, which ultimately led to finding the “Mother of the MP3.” I’m composing pieces from all of the tracks that were used in those tests – the most famous being Tracy Chapman’s “Fast Car” from 1988 – and will be releasing that material in the near future.