Editor's note: When Jonathan Scull reviewed the Shun Mook devices back in 1994, he unleashed a hailstorm of controversy that continues to this day. Below is his original report along with some of the follow-up articles and fallout.
Not that long ago, digital audio was considered perfect if all the bits could be stored and retrieved without data errors. If the data coming off the disc were the same as what went on the disc, how could there be a sound-quality difference with the same digital/analog converter? This "bits is bits" mentality scoffs at sonic differences between CD transports, digital interfaces, and CD tweaks. Because none of these products or devices affects the pattern of ones and zeros recovered from the disc, any differences must be purely in the listener's imagination. After all, they argued, a copy of a computer program runs just as well as the original.
"Equipment Reports," "Record Reviews," "Letters," "Industry Update," "Sam's Space," "As We See It," "The Final Word"---I read and enjoy them all. But the section of Stereophile I especially look forward to reading is "Manufacturers' Comments." How is the manufacturer going to respond to a review that's considerably less than 100% positive? Can they take criticism gracefully, or do they have an attitude? If I were a consumer considering purchase of one of their products, would their comments convince me that they'd be a good company to deal with? Are they uptight beyond reason, or do they have a sense of humor? Can they respond to a positive review without gloating?
Editor's Note: This article is now only available as an Audio Engineering Society preprint, under the title "The Role of Critical Listening in Evaluating Audio Equipment Quality," preprint number 3176. The price is $4.00 for AES members, and $5.00 for non-members; it can be ordered (currently on paper only, not as a downloadable pdf) by entering the number in the appropriate field on the preprint search page at the AES website.
"My system has great imaging!" "I can hear sound coming from beyond my speakers." "The depth image in my system goes back at least 20 feet." Yes, we audiophiles are proud of our imaging (footnote 1), and we've worked hard to get it. My back is still aching from the last time I tweaked my speakers until the image was just right.
It is always a matter of great interest when a difficult question, in this case the audibility of differences between amplifiers, is put to an empirical test. When the question is tested by such intelligent, knowledgeable, and unbiased investigators as John Atkinson and Will Hammond (see the July issue of Stereophile, Vol.12 No.7, p.5, the interest is even greater. Unfortunately, when the test turns out to have been flawed by errors in design and in use of statistics, as was the case here, the disappointment is also even greater.
Like many Stereophile readers, I have often sped home from a concert to fire up the audio system and then, to the sore vexation of my wife and guests, spent the rest of the evening plunged in the morbid contemplation of what, exactly, was missing.
Alright already, quit shoving. I know I don't belong here. This magazine already has a place for manufacturers---in the back, where those large egos are squeezed into small column inches so they can't hurt you. Not that I'm exactly proud of my job. On social occasions, if pressed as to my profession, I will usually admit to some honest toil such as mortician or hodcarrier. Speaker design is downright devious work. As proof, examine the specifications for the 1376 models in Audio's 1988 equipment directory. Much of this data, when compared with each described system's real-world performance, looks like Joe Isuzu wrote it on a bad day.
Editor's Introduction: Stereophile's "Recommended Components" feature is, as I am sure you will have guessed, produced by a committee. The reviews are studied, the reviewers polled to verify the continued validity, the merits and demerits of specific pieces of equipment are discussed or, rather, argued over at length by JGH, JA, and LA, and out of the whole business emerges the "truth." But, as with the findings of any committee, what is presented as a consensus will have significant undertows and countercurrents of opinion; if these are very strong, a "Minority Report" is often also produced. Such has been the case this time, concerning loudspeakers.
Editor's Note: In 1985 and 1986, an argumentative thread ran through Stereophile's pages, discussing the benefits or lack of double-blind testing methods in audio component reviewing, triggered by J. Gordon Holt's review of the ABX Comparator. As this debate is still raging nearly 15 years later, we present here the entire discussion that bounced back and forth between the magazine's "Letters" section and features articles. It was kicked off by a letter from C.J. Huss that appeared in Vol.8 No.5.—John Atkinson