U.Va. Computer Science Grad Student Develops ‘Musical Heart’

A University of Virginia graduate student has developed a biofeedback-based system that helps smartphones select music that will help get their owners’ heart pumping during exercise, or slow it down when they want to cool down or relax.

“Whether I am driving, jogging, traveling or relaxing – I never find the appropriate music to listen to,” said Shahriar Nirjon, a doctoral student in computer science in the School of Engineering and Applied Science. “I believe there are many like me. The problem is: The heart wants to hear something, but our music player does not understand the need. My joy was in connecting them together – in a non-invasive and cost-effective way.”

Called “Musical Heart,” the system “brings together wellness and entertainment,” Nirjon said.

Musical Heart works by merging a microphone that detects the pulse in arteries in the ear with earphones that bring in music from a playlist on a smartphone. An app selects tunes that optimize the heart rate of an individual user based on a given activity, whether running, walking or relaxing – playing fast-paced music for hard workouts, and slowing the beat for cool-downs. An algorithm refines the music selection process of the system by storing heart rate data and calculating the effects of selected music on the rate. Over time, it improves music selections to optimize the user’s heart rate.

Musical Heart uses a pair of specially built earphones equipped with tiny sensors to continuously monitor the user’s heart rate and activity level. That physiological information, along with contextual information, is then sent to a remote server, which provides dynamic music suggestions to help the user maintain a target heart rate.

The system learns, in essence, to select music that will have a desired effect on heart rate customized to the individual user, based on the effects of past music selections on the heart rate. In addition to helping to optimize heart rate for various activities, Musical Heart also can select music that helps the user relax.

“We’ve designed Musical Heart to be convenient, non-invasive, personalized and low cost,” Nirjon said.

Most portable heart monitors, he said, are not personalized to the user, meaning they don’t provide the biofeedback feature of Musical Heart. They can be inconvenient to use, requiring bulky watch-like monitors, sometimes with a chest strap, and can range in price from $80 to $400.

The Musical Heart system, which is not on the market, may cost about $20, Nirjon said, and is more than simply a monitor because of its music selection feature.

Nirjon will present his system at the 10th Association for Computing Machinery Conference on Embedded Networked Sensor Systems (SenSys 2012), being held this week in Toronto – the most important annual meeting in the field, he said. His paper describing the system has been accepted for publication in the conference’s proceedings.

Nirjon’s area of expertise is in smartphone sensing and networked embedded systems. Musical Heart is one of his research projects and will be a part of his Ph.D. thesis. He works with computer science professor John A. Stankovic, who runs a research group that bridges health and computer science. The industry partner on this project was Microsoft Research Asia (Beijing).

Nirjon currently is working on a generic acoustic sensing platform for smartphones that hosts a suite of apps – one of which is the Musical Heart. He does not currently have a marketing plan, but may develop one in the future.