Deeper Meanings Letters part 7

Nevertheless, I regard Harley's argument—that blind tests are "worthless" because they change more than one variable at a time—to be entirely specious. His alternative is to change all possible variables and then assert that component "A" is more musical than "B." Given the nature of the procedure and its reliance on phenomena that cannot be measured or quantified, if another evaluator reached the opposite conclusion would it not be equally valid? If several evaluators reached the same conclusion and most others found no differences, then I can confidently assert that a simple blind test would have revealed this statistically significant result, but with low confidence.

Harley states that there are audio phenomena that will forever defy quantitative measurement, and that attempts to quantify these phenomena automatically perturb the system sufficiently to invalidate the results—sort of an audio uncertainty principal. Despite his claim to the contrary, his beliefs and arguments are in direct violation of the basic scientific premise that nature operates using immutable principles; and that all physical phenomena are fundamentally capable of being understood, quantitatively modeled, and the model used to make predictions that can be tested by precise, objective, repeatable observation and measurement.

Although many observable phenomena remain unexplained and perhaps have not been quantitatively measured, the scientific endeavor rests on the assumption (belief) that they are subject to rational explanation and measurement. This is an old and continuing debate. Its most recent expression has been the controversy raised by Roger Penrose's book The Emperor's New Mind, in which he asserts that the human mind contains something that will make its capability always superior to that of a computer; that is, true artificial intelligence can never be achieved. He may be right, but he stands essentially alone among scientists and computer experts (footnote 6).

Harley's objection to blind testing repudiates a fundamental tenet of scientific inquiry—the necessity of falsifiability. Any "real" observable phenomenon must be repeatable and must be amenable to tests that could falsify its validity. For strict mathematical models (ie, theories such as special relativity, quantum mechanics, and electromagnetism) and semi-quantitative models such as evolution, this means providing predictions that can be tested. For hypotheses and conjectures it requires qualitative predictions that can be verified (or falsified), often by statistically valid blind testing.

Harley is free to maintain his own belief (which, as I argue, is fundamentally anti-science), as I am mine, but I suspect that we will continue to politely disagree. As I have shown, his belief contains all of the elements present in "Scientific Creationism." It involves the presence of unmeasurable and non-quantifiable phenomena; the employment of methods that cannot be reproduced, explained, or quantified; and the repudiation of the only known method of testing suitable for detecting and measuring the subtle (and not so subtle?) differences claimed to be present. Furthermore, he flatly denies the validity of the only means of verification that would lead to scientific credibility—falsifiability achieved through the blind test. If he could devise a legitimate falsifiability test and carry it out with positive results, I would probably be an early convert despite the presence of the remaining problems.

None of this denies the validity of the musical experience. When I attended the National Symphony Orchestra's performance of Britten's War Requiem, I was profoundly moved, but it would have been entirely impossible for me to have provided a useful review of the quality of the interpretation or the performance other than to say that I was not aware of any glaring flaws. By contrast, I can readily identify differences among the three different recorded performances that I own, although I cannot achieve the same musical experience that I felt during the NSO performance. I can easily discern differences in the quality of speaker systems, and to a lesser degree judge which is the more "natural," although the room response seems to weigh heavily in this evaluation. I have been able to detect subtle differences in a variety of equipment, including phono cartridges and my old Heathkit amps vs my present Hafler amp, but in these cases measurements have been the more secure means of detecting and quantifying the differences.

Scientists certainly can be arrogant in their adherence to strongly held beliefs, as the example of Ernest Lawrence indicates. I can provide numerous examples, but with rare exceptions, the scientist can be persuaded by the results of theory and experiment. Yes, he does adhere to the scientific method and strays beyond it to his peril. Harley's footnote 6 sums up the issue precisely. "It is very dangerous to argue scientifically indefensible positions with scientists if one wants to remain a member of the scientific community." Exactly correct. Arguments concerning subjective experiences are simply not part of the agenda of science. To insist that they be included as Harley does is simply wrongheaded and will not be accepted.

To do so would set back the clock several hundred years and open up Pandora's box, releasing a torrent of claims of every imaginable type, each of which would find sufficient adherents to meet Harley's criteria: 1) the effect is not measurable by scientific techniques, and is outside the range of scientific methodology; 2) practitioners can achieve the effect using techniques that cannot be explained by existing theories and laws of nature; and 3) especially skilled individuals can observe the phenomena, ipso facto it is real, but by nature it is not repeatable on demand and blind testing is irrelevant.

Harley also exhibits two other characteristics that I find disturbing. First, when faced with evidence that is not favorable, he immediately finds a facile explanation for the failure (false reporting of results, bias on the part of the testers, inapplicability of the test); and second, he attacks the critics by accusing them of being biased because they refuse to accept his arguments. (I do not excuse the AES for being derisive and uncharitable in their attitude, but if the same arguments are used by audiophiles to support their position as have been used by proponents of astrology, N-rays, Hieronymus machines, and ESP, and the audiophile can offer no evidence other than his personal subjective experience, then the comparison is inevitable (footnote 7).

Harley also seems to argue that scientists do not listen and are incapable of having a musical experience. That is true arrogance. The musical experience is subjective and real, but attempting to quantify it is akin to reviewing Cage's 20 minutes of silence. He may tell me that a component sounds to him more musical than another component that he listened to at another time. If he describes those differences in terms such as "palpability," "soundstage," "grain," "bloom," etc., we communicate only if we share an objective understanding of the meaning of these terms. I am able to accomplish this goal (most of the time) in most fields of art such as painting, sculpture, and photography, where technical measurements do not exist. But Harley indulges in self-contradiction by insisting on the non-measurability of these characteristics and then proceeding to define and quantify them for application by others.

It is this attempt to quantify an essentially subjective threshold effect, that has not been shown to be either repeatable or generally detectable, that causes the difficulty. Numerous examples are known from science itself (Blondlot's N rays, Davis and Barns's electron-capture measurements, and Gurwitsch's mitogenetic rays are good examples). In every one of the instances with which I am familiar, the reported phenomenon was found to be the result of highly subjective, biased, or simply erroneous observations made at the threshold of visual detectability. The phenomenon vanished when observations were made in a more rigorous fashion. It is not surprising that scientists look with suspicion on such observations—they have had their fingers burned too often by not applying good standards. "Cold fusion" is the most recent example.

The most celebrated case is the work of J. B. Rhine, whose efforts to demonstrate the reality of extrasensory perception is a classic instance of pathological science. Most commonly, the problem has been that whatever effect is being measured is near the threshold of visual detection; threshold audio effects are subject to the same problems.

Harley's insistence that science accept listening tests as an expansion of testing is simply unacceptable. When the measurements end science has completed its task. If the microphones are imperfect and their placement critical, if the amplifiers produce results either bizarre or beautiful to the user, if the ADC is highly nonlinear, then the recording engineer will use his skills to achieve a result that he may consider acceptable or wonderful. One reviewer may groan, another applaud. If a speaker sounds good or bad or realistic or whatever, I may like it, I may be able to correlate some of what I hear with what has been measured, or I may relate this to my audio test expert friend and ask him if he can make other kinds of measurements that may correlate with what I hear. If he cannot, so be it. Others may have entirely different perceptions of the speaker.

In the past, the "what is good" problem was endemic to speakers. I well remember that musicality in the minds of many involved lots of boomy bass. My speakers lacked it and were roundly criticized. What is good changes with time, and probably with the age of the individual and [his or her] total experience. If the Wadia 2000 sounds musical and tests badly, the scientist wants to know why and to find a way of reproducing the effect. But first he needs to know if a significant fraction of qualified listeners can observe the effect. If you refuse him that step and tell him that the effect is unmeasurable, he can only shrug and go on to other things. Listeners can hear musical effects, and that ability has been roughly quantified. If the effect is essentially magic as defined above, it is out of the scientist's realm.

Harley explicitly attacks the basic premise on which all of science rests (and by implication the existence of the equipment to which he listens, which is an expression of that science) when he derides this premise: "Consideration of any event outside the measurable and repeatable is to stray from the scientific method and thus away from the truth." Almost right. The gravitational bending of light was nearly unmeasurable in 1919. It was attempted because it had been predicted. But it was certainly repeatable. On the other hand there are numerous examples of hypotheses that were not testable when first presented. They usually are discussed and studied (unless very unconventional), and in most cases they eventually stand or fall based on measurements.

Subtleties do exist. But I firmly believe that they can be measured, either now or in the future, and that measurements will (ultimately) be a more reliable indication of quality for most people than subjective (and inherently biased) listening tests. Where subtle effects are suspected, and controlled measurements cannot be done, the double-blind test is the accepted means of detecting correlations. Thus we have the connection between smoking and heart disease and many other results that would have been difficult to establish by laboratory measurements. Without such tests the scientist has nothing to contribute and will rightly regard equipment reviews based primarily on listening "tests" as being in the same category as the critical review of a play, concert, or a recorded performance.

I freely admit that, despite intense concentration and effort, I have failed to detect or respond to the subtle differences that Harley insists are present. He will probably argue that I have the usual scientific bias that prevents me from observing the obvious. But even if I believed Harley's statements concerning the Wadia 2000, there is little or nothing that I could do. I haven't a clue as to why the effects exist or where to start to look for an answer. Do I abandon Maxwell's equations and the Ebers-Moll model of transistors and attempt, without a shred of useful data, to develop a whole class of new models of electronic components and the way they function? To do any of the latter I need to know how human hearing functions. To do that I must start with blind testing, or at least the kind of sterile laboratory testing that Harley decries. But if the effect is nonphysical, non-measurable, and non-quantifiable, what do I do? Even magic in the many fantasy stories that I have read (Tolkien, Anthony, Kurtz, Chalker, etc.) is based on some kind of logic.

Footnote 6: For a good discussion of the nature of consciousness and the attempts by computer and AI scientists to replicate at least some of its aspects, see George Johnson's Machinery of the Mind (Tempus, 1986). (My thanks to Bill Sommerwerck for giving me a copy of this stimulating book.)—JA

Footnote 7: All of the examples cited, along with a number of others, disappeared when subjected to rudimentary blind tests. The ESP issue hands on in various guises because adherents argue that it is unmeasurable by scientific methods. They usually reject the blind test as invalid. Rhine actually selected his data based on positively correlated card-guessing runs, apparently in ignorance of the statistics of Markov chains. He simply did not report negatively correlated runs or the overall averages.