Think Pieces

George Reisch's September 2000 "Undercurrents" column addressed issues raised by Mr. Salinger

Pornography or audio journalism?

I knew that someone would find the Achilles' heel in my musings on medicine and subjective testing in May's "Undercurrents." As Ross Salinger notes in his letter in this issue, the parallel I drew between medicine and audio goes only so far. To be admitted to the halls of medicine, alternative medical techniques are tested with blind tests. But subjective audiophiles usually eschew blind tests.

For Mr. Salinger, that's a problem, and he generously outlines a new kind of blind test to challenge me and other audiophiles: "Just have someone substitute cables or any other component without his knowledge in 50 trials over a two-month period." Then, he suggests, we would know "the facts about subjective listening tests," at least as far as my slightly alloyed golden ears are concerned. Mr. Salinger believes that I would have "no idea at all of what's been changed in his system."

The proposal cleverly circumvents a standard criticism of blind tests. I can't complain about the fact that I missed, say, the differences between 99.9% and 99.999% pure copper cables because I was distracted by the curtains, the program material, or unfamiliar components in the test setup. I'd stay right at home with my beloved audio system. But I'd still have to keep a "daily diary" in which I would, in effect, review the performance of my system. Never knowing for sure if Mr. Salinger's blind-test elves had swapped a component or cable or tweak while I was out, I'd have to listen critically and make an entry in my diary during (or just after) every listening session.

Mr. Salinger doesn't think this is problematic. How, he asks, would "not knowing in advance" what has been changed in my system make me less able to hear the sonic effects that most audiophiles attribute to tweaks or cable-swaps? I have an answer for that, but it's not a simple one. The saying goes, "for those who believe, no proof is necessary; for those who don't, none is possible." Fortunately, the saying is about God, not blind tests. So here's an argument (proof's underrated cousin) to show that blind tests stink—even Mr. Salinger's audiophile-friendly version.

For argument's sake, consider a sensuous experience different from listening to music: making love, having sex, whatever you want to call it. Suppose some connoisseurs claim that very small factors—say, the precise angle of a haircut, the fiber content of clothes, eating certain foods, using certain soaps, or other small tweaks—make small but important differences in their amorous experiences. For them, they claim, it matters whether their partner uses 99.9% vs 99.999% pure soap.

The arguments would be familiar. Skeptics waving the banner of science would say that these effects are illusions. Science understands a lot about sexual behavior, and these variables just don't enter the picture. Nor do they find that their own libidos start percolating when their spouses or lovers experiment with the tweaks in question. But they're open-minded, even to the idea that they work for only some people ("golden-..."). They'll believe, they say, when the alleged effects of these tweaks are shown to hold up under blind tests.

At first, the connoisseurs are enthused. "Blindfolds! Great idea!" But then the reality sinks in. Following Mr. Salinger's proposal, they will live their lives as they normally do while elves secretly and systematically adjust the variables in question. They will keep a diary describing their experiences. As best they can, they must say whether one of these performance-enhancing variables has been changed and, if possible, specify what that variable is. After two months, the skeptics will collect the diaries, analyze the results, and look for statistically significant correlations.

This test would be just about useless. Assuming they're wearing any, put yourself in the subject's shoes. During the test, you'll have something new on your mind: "How am I doing?" you'll ask yourself. "Is this particular sensation new, or something I've experienced before? Yes, it's different, I think. I'll have to find a word for that later on when I write in my diary. Oh, and what's that fragrance—is my partner using a different soap tonight? Hmmm. Not sure about that one. But something seems different. Could this be a 'statistically significant correlation'?" And so on...

If such an interior monologue doesn't strike you as the opposite of an aphrodisiac, then you're unique, or too young to be reading this magazine. This test would put a damper on the phenomena it was designed to measure. When those diaries are collected and analyzed, they would have entries like "not quite as much fun...not going so well since this lousy test began."