By recreating in a computer both the real-world physics of a clarinet and the physics of a clarinet player, researchers have developed a way to compress 20 minutes of a solo into a meager 1 kilobyte (Kb) of file space.
The term "Internet music" usually applies to electronic files that are downloaded, ripped, streamed, burned and shared from one device to another. The popular compression program known as MP3 makes relatively large music files smaller so they can be downloaded faster and easily stored on playing devices. That all works pretty well for listening to and mobilizing recorded music.
But what if you want to actually play music over the Internet—at the same time with someone else who is located somewhere else? Well, right now, playing a duet with a distant partner would be more like instant messaging, only not nearly as instant.
Although researchers are trying to fix that, it's no small problem. The dynamics of music, especially when performed with others, challenge the most-discriminating human senses. And of course, timing is everything, both for the played notes and the silent spaces in between. Even in large concert halls, millisecond delays occur when sound travels from, say, the first violin player to the tuba player, and especially from the orchestra to the audience.
So imagine trying to cram even simple music pieces into files small enough to shove back and forth instantaneously over the Internet so distant performers can acoustically interact as though they were in the same room. That's precisely what Mark Bocko and his team at the University of Rochester are up against. They call it "tele-immersion," and a cadre of clever scientists and humanities researchers are developing the technology to bring together not only musicians, but dancers, artists, surgeons, and students and teachers in a shared, if geographically apart, reality.
A physicist by training, Bocko and his graduate students in Rochester's Department of Electrical and Computer Engineering are working with music professor Dave Headlam to solve the problem by getting back to basics—that is, capitalizing on the flesh-and-blood limitations of mortal humans.
"Humans can manipulate their tongue, breath, and fingers only so fast, said Bocko, "so in theory we shouldn't really have to measure the music many thousands of times a second like we do on a CD. As a result, I think we may have found the absolute least amount of data needed to reproduce a piece of music."
By recreating in a computer both the real-world physics of a clarinet and the physics of a clarinet player, the group as made a small but important step toward overcoming the slowness or "latency" barrier of two-way transmission of audio files over the Internet. At a recent scientific meeting, Bocko announced his team's success in compressing 20 minutes of a clarinet solo into a meager 1 kilobyte (Kb) of file space. Bocko says that's about 1,000 times smaller than a regular MP3 file. By comparison, a blank Microsoft Word page consumes 24 Kb.
The researchers measured every aspect of a clarinet that affects its sound—from the backpressure in the mouthpiece for every different fingering, to the way sound radiates from the instrument. They then built a computer model of the clarinet, and the result is a virtual instrument built entirely from real-world acoustical measurements. In replaying the music, a computer literally reproduces the original performance based on everything it knows about clarinets and clarinet playing.
The team then set about creating a virtual player for the virtual clarinet. They modeled how a clarinet player interacts with the instrument including the fingerings, the force of breath, and the pressure of the player's lips to determine how they would affect the response of the virtual clarinet. Then, says Bocko, it's a matter of letting the computer "listen" to a real clarinet performance to infer and record the various actions required to create a specific sound. The original sound is then reproduced by feeding the record of the player's actions back into the computer model.
"Maybe the future of music recording lies in reproducing performers and not recording them," says Bocko. For now, the technique handles only one instrument at a time, but combined with another method for separating multiple instruments in a mix, the two inventions can be combined to produce a very compact recording.
Alas, the reproduction is not yet flawless—it's modeled after human beings after all—but the researchers say it's getting close. Bocko believes that the quality will continue to improve as the acoustic measurements and sound reproductions become more accurate, and he says this process may represent the maximum possible data compression of music.
This research, which was presented at the International Conference on Acoustics, Speech and Signal Processing earlier this month, is funded by the National Science Foundation.
—Leslie Fink/NSF
This report is provided by the National Science Foundation, an independent federal agency that supports fundamental research and education across all fields of science and engineering, in partnership with U.S. News and World Report.
