Quote:
In an earlier posting, I asked why blind testing was such a hot button for some audiophiles? As I mentioned in the HE 2005 Debate -- see http://www.stereophile.com/news/050905debate -- my skepticism about the efficacy of such tests is based both on a large amount of experience of such tests and my scientific education leading me to question the methodology adopted by so many of the published tests. By contrast, with a few notable exceptions, I have found those who respond most vocally to my criticisms and have implicit faith in blind tests have neither a scientific education nor any experience of blind testing.

I was reminded of this when, retrieving articles to post in our free on-line archives, I came across the following passage I had written years ago:

"This uncritical faith in Science is examined in Harry Collins's and Trevor Pinch's <I>The Golem</I> (Cambridge University Press, 1993). Professors Collins and Pinch look at formal scientific method as practised in a small number of classic experiments -- Michaelson & Morley's proof for the nonexistence of the ether, for example -- and conclude that "objectivity" is more intimately linked with society's and scientists' expectations and needs than is generally appreciated. They also examine the general public's flip-flopping between distrust of, and blind adulation for, Science. Regarding the latter, Collins and Pinch point out on p.143 that "It is no coincidence that those who feel most certain of their grip on scientific method have rarely worked on the frontiers of science themselves."

Interesting, no?

John Atkinson
Editor, Stereophile

Yes, the process of creating new science (or mathematics) often does not resemble the way science and math are taught in school. Many engineering students (and a significant fraction of science students as well) are not familiar with the exploratory nature of thought involved at the frontiers of knowledge. A number of years ago, I was invited by the President of one of the nation's top five ranked engineering schools to consult with the math professors about teaching math in a motivating way, rather than in a wrote memorization way that made the calculus course into a filter that caused many students to change majors. It turned out that the professors wanted to do this, and my role was restricted to advocating the funding to do so. It did emphasize for me, however, that the nature of creative thought in science remains a mystery for many students and professionals, as well for many members of the general public.

It will always be true that "extraordinary claims require extraordinary evidence." It is also true that experience with changes in scientific thought teaches a humility about the limits to our knowledge. I am bothered whenever a scientist (in a non-religious context) says "I believe" a particular hypothesis. I would prefer to say that I think the probability is high (or low). I am experienced enough by now to question the current state of knowledge in some cases. I often find the current view to be well founded, but sometimes it is not, which may lead to innovation.

In the case of audio DBT, it is a very useful tool that has (I am told!) apparently been used to great effect, for example, in the design of Revel loudspeakers. But, when DBT is used as evidence that some effect does not exist, then I take issue with that claim. I do form my own impressions and opinions of probability based upon a combination of experience and physical plausibility. I expect amplifiers differences to become audible under some conditions due to interactions with speaker loads, for example. When I hear that humidity changes the sound, I do see a theoretical basis in the air density change, but I doubt its significance, not having heard it myself. I remain open to the observation of a humidity effect, however, if evidence does arise. In cases such as this, high fidelity system modeling (the system includes the recording studio and the ear-brain of the listener) would be instructive, but I have not heard of a significant effort in that direction.

Given our current lack of knowledge in the pursuit of audio fidelity, we are all participants in the exploratory adventure of creating a better knowledge base. That means we can have fun trying things like DSP and ion speakers. Someday, if it ever happens that everything is known, it may be better for music appreciation, but it will be less interesting for technologists. The question that really interest me is: "Am I happy with the sound of my system?" The answer is no, emphatically. So, what to try next? An audiophile opportunity!

Just like any political issue, I do not agree with either of the extremes at on each side of this debate.

Fundamentalist subjectivists are as wide eyed and off planet as the beady eyed DBT zealots.

The DBT guys don't sound like they allow any fun, and the subjectivist reviewers make round about admissions that they would fail any listening test more rigorous than having price sheet info on an item right in front of them. (No offense.)

I will always maintain that the path to audio Nirvana is best travelled by chosing the path of most entertainment.

Blind testing is done by everyone, not with comparators and gear, but by audiophile buddies changing cables, placing a different amp in the path, anything they can think of to try and see what their buddies think. Stereophile has a great resource for doing this, but I can understand the apparent fear of "being found out." All subjectivist leaders share the same fear and gather the wagons to condemn objectivism by identifying its extreme to attack the whole notion. Sly, but disingenuous.

I tell you what, as a consumer, I would think I am worth a magazine getting two of something - cable, cartridge, amps, whatever...and having a reviewer compare different samples of the same device or having two reviewers review the same device.

What would be the harm in a reviewer not knowing the brand name or cost of a cable he is reviewing?

What is the threat of two reviewers independently reviewing the same device?

What, I ask you, is wrong with telling a reviewer that he may or may not have a new cable in his listening chain and having him describe what he hears after any given "change?"

Reviewers are pros, they should not have performance anxiety - these situations should find them champing at the bit to get to it. They talk about honed listening skills and dramatic and life changing devices, they should relish this opportunity to tell us just how revealing their favorite recordings are under these circumstances.

A reviewer who is so acutely aware of sampling rates in the discs he has engineered should be able to quickly pop his favorite discs into his favorite transport and tell us what he hears from a potentially unknown or unchanged downstream device playing discs he has made.

This is what we all do with our test discs and friends' systems. Every hi-fi club does it, too. It's FUN. It's vital. But it's mostly shunned by reviewers who validate this shunning by attacking the extreme DBT faction.

Nice move, but I don't buy the reasoning.

I am frustrated that you feel so threatened by ANY blinding that you would not be willing to try and give this more of a go here and there.

Let what's left of your hair down and spend one issue having a blast enjoying what everyone else is able to do!

I bet you'd double your average sales figures for that issue!

Tell Sam Tellig that I bet Lars would say to go for it, too.

Quote:

I tell you what, as a consumer, I would think I am worth a magazine getting two of something - cable, cartridge, amps, whatever...and having a reviewer compare different samples of the same device or having two reviewers review the same device.

I don't see the need for blind testing, but Buddha has an interesting point in comparing multiple samples. Even more interesting would be multiple competing components. The magazine Car and Driver often does comparison tests. It might be 8 comparable cars in some such category as mid-priced sports sedans. They might have 8 editors rotate driving the cars, and then each would rate the order of preference. The editor-in-chief (or someone) then compiles the overall rankings. While YMMV, and I don't always agree with the order of the ranking, I find those comparison tests quite entertaining, as well as educational. Those tests are not blind, but very revealing in some cases. I do wish they would mix price classes more often. They did run two companion articles a few years ago about the best handling car under $30K, and the best handling car over $30K (it would have to be $40K now!). The best handling car under $30K, if I remember correctly, was the Honda Prelude, which was then inserted into the best handling car over $30K test, and it finished mid-pack among some very expensive machines. I understand that this may be beyond the resources of stereophile, but as a reader, I would enjoy comparison testing.

Human, or humanoid? There were some blind men in a cave, or, some men who had never seen anything in a place where nobody ever saw anything, or didn't see enough to be sure, or didn't remember, whatever ...

I was talking to the designer/owner of one of the companies that has a component on the Stereophile recommended components list. He happens to, in addition to the recommended component, make a very high end amplifier line that is relatively unknown. I have listened to it and it is fabulous (and also out of my price range).

Early in the design phase of his amplifiers, he sent out beta test units. The input was less positive than what he had expected. The consistent complaint that he received was that the amplifier was too small and it would never sell. Now engineers are funny sorts of folks. They are all about efficiency. Why make an amplifier huge, when small works just as well?

So, he collapsed to logic and common sense, and made the amplifier way bigger and heavier than it needed to be. The amplifier is the exact same amplifier, but now everyone is happy.

So, there you have it. People are buying audio products based on factors that have NOTHING to do with the only thing that is of paramount importance, the sound of the unit. I simply do not understand it.

Is not the goal of a true audiophile to really enjoy music? In scientific testing, one of the paramount rules is to eliminate all extraneous factors that could effect the outcome of the test. To determine if a change in A effects B, you must first make sure that there is not a changing C out there that is also effecting B. This also happens to be a cornerstone of logic. Obviously, whether I like it or not, I'm effected by my other senses. So why in the world would I want them to come into play? When I'm listening intently to music, I close my eyes and listen. Some of my most enjoyable listening is in a completely dark room. In other words, I blind myself.

Science is the basis for all sound decision making. No pun intended.

Quote:
So, he collapsed to logic and common sense, and made the amplifier way bigger and heavier than it needed to be. The amplifier is the exact same amplifier, but now everyone is happy.

Isn't it possible that the bigger, heavier version affected the sound in some beneficial way through dampening vibration via its increased mass, or ended up being better shielded as a result of these cosmetic changes? While the circuitry sported no alterations, I would suggest this might no longer be the same amplifier.

Quote:

What would be the harm in a reviewer not knowing the brand name or cost of a cable he is reviewing?

What is the threat of two reviewers independently reviewing the same device?

What, I ask you, is wrong with telling a reviewer that he may or may not have a new cable in his listening chain and having him describe what he hears after any given "change?"

1. Practicality. This could only be accomplished with a central system that is set up before the reviewer would visit or by having someone slip in and modify the reviewer's own system without his knowledge or involvement. In the latter instance, some method of camouflage would also be required. Hey, nobody messes with my stuff but me!

2. We've done it.

3. See #1.

Kal

Maybe I was not clear enough. This was not from casual observation of circumstance. That would lead to an unscientific conclusion. He told me himself that the amplifier was bigger for cosmetic reasons only. I could tell that it frustrated him. He said that it measured and sounded EXACTLY the same.

Given that you are dismayed that buyers of audio products make purchase decisions for a variety of reasons - some unrelated to the primary purpose of the product, I'm curious about what factors influence you when you buy an automobile or dinner at a restaurant. Is there more involved than efficient transportation and nutritional value?