Blind Listening

John Atkinson sets the stage
Nothing seems to polarize people as much as the vexed question concerning the importance of audible differences between amplifiers. If you think there are subjective differences, you're an audiophile; if you don't, you're not. And as any glance at an appropriate issue of Consumer Reportsthe publication for non-audiophiles—will confirm, the established wisdom is that once the price of an amplifier or receiver crosses a certain threshold, any further improvement in sound quality becomes irrelevant, in that it puts the price up for no apparent gain. In other words, when it comes to amplification, there is such a thing as being "too" good. Yet, as a reader of this magazine, I would expect that not only have you been exposed to real subjective quality differences between amplifiers that Consumer Reports would regard as sounding identical, you have made purchasing decisions made on the basis of hearing such differences.

It has often been said that the only way to resolve this apparent dichotomy is to use carefully controlled blind listening tests, where the listener does not know what he or she is listening to. In this manner, imaginary differences should fall away, leaving real differences that can then be correlated with objective measurements. Unfortunately, as you will have noted, for example, from David Clark's infamous blind amplifier test in Stereo Review (footnote 1), it seems that with such blind listening tests, all perceived subjective differences between amplifiers (apart from those due to level, absolute polarity, and amplitude-response differences) fall away. The conclusion then drawn by some observers is that, indeed, once above a certain performance threshold, amplifiers do sound alike.

But when you have taken part in a number of these blind tests and experienced how two amplifiers you know from personal experience to sound extremely different can still fail to be identified under blind conditions, then perhaps an alternative hypothesis is called for: that the very procedure of a blind listening test can conceal small but real subjective differences. Having taken part in quite a number of such blind tests, I have become convinced of the truth in this hypothesis. Over 10 years ago, for example, I failed to distinguish a Quad 405 from a Naim NAP250 or a TVA tube amplifier in such a blind test organized by Martin Colloms (footnote 2). Convinced by these results of the validity in the Consumer Reports philosophy, I consequently sold my exotic and expensive Lecson power amplifier with which I had been very happy and bought a much cheaper Quad 405—the biggest mistake of my audiophile career!

Some amplifiers which cannot be distinguished reliably under formal blind conditions do not sound similar over lengthy listening in more familiar and relaxed circumstances.

There is also the fact that the ability to reliably hear differences between hi-fi components varies considerably from person to person. There is no doubt in my mind that it is, to a large degree, a learned skill. If you cast your mind back to the January 1989 issue (p.63), you will recall that I reported on a large-scale blind listening test carried out by David Clark at last November's Audio Engineering Society convention that, among other things, involved amplifiers. As expected, when all the results from all the listeners were lumped together, it appeared that the three amplifiers under test sounded identical; ie, the listeners as a whole failed to identify when one amplifier had been changed for another in a paired comparison or when it had remained the same. Looking at individual results more closely, however, suggested that there was some identification. As The Abso!ute Sound's Michael Fremer reports in this month's "Letters" section, no only did he identify correctly five times out of five whether one amplifier was the same as the next or different, he also volunteered identifications of the amplifiers under test—Crown, Threshold, and VTL—and got that right four times out of five.

"Not statistically significant!" came back the reply from non-audiophiles. Even when I added my score of four out of five to Michael's five correct to give an apparently convincing nine correct out of ten identifications, the response was "You're just 'lucky coins' and if you did the test all over again, you wouldn't do as well the next time."

That, of course, is conjecture, and with the tests long since over, is impossible to prove one way or the other. But I was sufficiently impressed with David Clark's methodology that I thought it would be worthwhile repeating the tests at Stereophile's High End Hi-Fi Show in San Mateo last April, using the visitors as test subjects. And whereas only about 130 listeners had participated in the AES tests, I could expect about 500 listeners to take part in my tests, which would endow the results with greater statistical power when it came to determining whether the listeners had identified the amplifiers under blind conditions.

Footnote 1: Stereo Review, January 1987, p.78

Footnote 2: HFN/RR, November 1978, p.110.