Audio Realism Breakthrough

Audiophiles of every stripe know that achieving a realistic soundstage is maddeningly elusive—some would say impossible. When a system is right for some recordings, it's far off for others.

It's especially difficult with only a single pair of loudspeakers, due to huge variations in room acoustics and the almost always unknown but essential details about how a recording was made. Binaural headphone recording and playback has always offered better spatial cues, sometimes with disturbingly realistic effects. Using a dummy head with microphones placed where human ears would be, recordists can capture enough lateral information to trick the brain into perceiving a three-dimensional soundfield when the recording is played back through headphones.

The effect is especially pronounced with sounds to the sides and rear of the listener, and less pronounced toward the front. Some technological artists have exploited binaural recordings with eerie results. The San Francisco Museum of Modern Art offers a tour that is both real and virtual, in which visitors don headphones and watch video playback on loaner camcorders as they follow the exact steps taken by artist Janet Cardiff as she walked through the building. At one point while you're climbing the stairs, someone who isn't there runs up and past you. It does make you wonder about the nature of perception and reality, a dominant theme in contemporary art.

Playing binaural recordings through loudspeakers in the open air doesn't translate, however—if anything, the sonic image is far worse than with recordings intended to be heard over speakers. Many audio engineers have theorized that a larger number of record/playback channels, including some for vertical cues, could re-create a much more realistic soundfield. Tomlinson Holman, developer of the THX system, believes that a "10.2" system, with ten dedicated full-range channels and two low-frequency effects channels, would be sufficient to do so. Implementing such a system is probably not a practicality even for the most fanatical audiophile.

An intensely realistic system with headphones appears to be feasible, however. On June 29, 2004, researchers at the University of California at Davis announced a breakthrough in "motion-tracked binaural sound" (MTB). The system "captures cues for direction, distance, and movement and the subtleties of natural, ambient sound that other systems don't," according to the announcement. Developed by Ralph Algazi, Richard Duda, and Dennis Thompson at the Interface Laboratory in the UC Davis Center for Image Processing and Integrated Computing (CIPIC), the patent-pending technique "uses off-the-shelf equipment that won't break the bank."

Although the "head-related transfer function" (or the way the head attenuates sounds from one direction to create spatial cues in the brain) has been recognized for decades, its only practical application has been in training airplane pilots.

UC Davis scientists tackled the age-old problem of capturing spatial cues in a novel way, leveraging computer technology not available to prior generations of researchers. MTB is said to incorporate more information about the space in which recordings are made, and makes possible pronounced changes in perspective when listeners use a specially designed headset.

"Conventional audio playback doesn't reflect how you hear in real life," Algazi said. "Your body, the shape of your head, and the room acoustics all affect how you hear." Another persistent playback problem is the necessity to keep one's head still, as any audiophile with extremely "beamy" loudspeakers can attest. Moving one's head while listening to traditional binaural recordings violates the illusion of realism. MTB uses multiple microphones (8 for voice, 16 for music) spaced around a head-sized ball or cylinder. Sound played back through headphones with a small tracking device mounted on top enables the playback equipment to follow head movements, mixing sound from different microphones and thereby maintaining a more realistic perspective.

The effect is said to allow listeners to "move their heads to locate a sound source, turn to face a person speaking, and tell when sound sources are nearby or far away. In addition, MTB captures the ambient sounds of the location, so you recognize the echoes in a church or the confines of a conference room."

"The system can capture the sound of instruments much more fully than a conventional single microphone. It captures changes in sound and the effects of a room in a way that is much closer to reality," Algazi said.

The MTB system was described as "something really wonderful," by Pablo Ortiz, chair of the Department of Music at UC Davis. "The thing that's interesting to me is the way that it records space," he stated. Live recordings could be a "major application" for the system, according to William Beck, a composer of electronic music and lecturer in the UC Davis Music Department. "The 'being there' feel is something people would really like," Beck said. Few audiophiles would argue with that.

The announcement didn€™t mention any commercial products that might incorporate MTB, but the application for a patent implies that it does have commercial potential. MTB could prove to be everything that high-rez free-air technology is not.

More info:
CIPIC Interface Laboratory
National Science Foundation