Challenges to Watermarking Continue in Wake of London Tests

The Secure Digital Music Initiative (SDMI) will soon move into Phase II of its evaluation of digital audio watermarking, following listening tests conducted in early July at Sony's Whitfield Street Studios in London and administered by Sony VP of engineering Malcolm Davidson. A soon-to-be-published report from Paul Jessop of the International Federation of Phonograph Industries reveals that the participants in the tests—almost all of them audio-industry professionals or journalists—averaged just slightly better than 50% in their abilities to detect the watermarks.

The Whitfield Street trials were roundly criticized by some who took part: for the selection of test material, which was said to be marred by analog tape hiss; some extraneous noise in the listening area from a laptop computer; and for the use of ABX testing, long a controversial topic in the industry. "Notoriously tough for the inexperienced, and very tiring," is how classical engineer Tony Faulkner of London's Green Room Productions described the test methodology.

Faulkner continues to be one of the most outspoken critics of watermarking. He had predicted that the Verance watermarking techniques selected by SDMI would become more audible, not less, with more revealing recordings than were used in previous trials. He vindicated himself in the London tests, scoring 75% correct on certain portions.

At the Whitfield Street trial, 27 individuals tried to identify watermarked recordings from four different record labels, in a total of 880 attempts. All the material was sampled at 24-bit/96kHz resolution. The results, as reported by Davidson and Jessop:

LabelMusic SelectionCorrectTotal Trials Percentage
UMG (Universal) Debussy (Solo Piano) 113 224 50.45%
BMG (Bertelsmann) Prokofiev (Orchestral) 114 224 50.89%
WMG (Warner) "Little SunFlower" (Pan Flute) 110 216 50.93%
SME (Sony) Trey Lorenz "Proper Thing" (Pop) 108 216 50.00%

The average audibility rate for the 880 attempts is 50.57%, according to Jessop. Faulkner is challenging the test interpretations put forth by the SDMI and IFPI that a 50% average score in the Whitfield Street trial indicates that the watermarks are inaudible. The figure—virtually the same as random chance—could just as easily be attributed to "lousy test methods, lousy source material, lousy non-standard listening room, lousy monitoring, [or the] ABX method," Faulkner stated in an e-mail after receiving the collated results. He will participate in panel discussions on the topic at the Audio Engineering Society convention in Los Angeles in late September.

Davidson, meanwhile, has announced that SDMI will move into its next testing phase, which will include music recorded and sampled at 24-bit/192kHz. Verance has already signed licensing agreements with several music companies. Potentially worth millions, the deals are risky only if the music-buying public rejects watermarked recordings.