Features

No cable is that good, no amplifier is perfect, no speaker is perfect. So while there is an absolute reference, nothing ever will be perfect, and in comes the subjectivity in choosing what compromises are more acceptable. But it still doesn't mean that it's totally subjective. To me the speakers that allow me the easiest time to forget the system, which to me is the purpose of a good hi-fi—to see past it and simply enjoy the music—point-source speakers do that the best for me. Planar speakers do it better for others. Most of you probably have an inclination one way or the other, yet as professionals in the community, we can't let that inclination get in the way of our judging product suitability for other people. Again, whether as store buyer or reviewer, this ideal person having an opinion on whether a planar is better than a point-source or a point-source is better than a planar is not a productive opinion.

The consumer can have that opinion; he's only going to buy one product. But it's not fruitful [for a reviewer to express that opinion]. The Audio Critic once declared that transformers were inherently better than headamps so you shouldn't buy a headamp, you should buy a transformer. This kind of opinion is one of the things that is most wrong with the reviewing community. There is this tendency on everybody's part—it's a male problem, it seems to me, about relating to these toys, this hardware—about wanting to have an opinion on it in ways that aren't appropriate or helpful to the actual opinions that're required: Is it good? Is it worth owning?

An opinion about whether a headamp is better than a transformer or vice versa is something that a manufacturer needs to have before he goes out to design a product, to have an inclination of what he thinks is a more productive route. For the consumer, for the reviewer, and for the store buyer, even knowing that it's a headamp or a transformer ahead of time is likely to prevent them from actually hearing the truth of the product. It's hard not to have these types of opinions—belt-drive is better than direct-drive, single-beam laser is better than triple-beam—but I think the hardest thing for anybody involved in hi-fi, whether it's at the Stereo Review level or the Stereophile level, is not letting what you know get in the way of using your ears in judging the product. It's tough, but it's something that should never be forgotten.

John Atkinson: But surely every opinion the reviewer has is analogous to the one concerning transformers and headamps? For example, his experience is going to tell him that with this kind of tweeter, it's unlikely he's going to like the speakers even before he sets them up because he's heard other products using that same tweeter. He therefore can't help but be preconditioned by his experience. If he wasn't preconditioned by his experience, then he probably wouldn't be any use as a reviewer anyway. So, how could someone forget everything he's experienced and learned when he approaches a new product?

Low: I hope this is part of the distinction between an amateur and a professional reviewer.

Dick Olsher: I think we all have a certain ideology which is based on experience and cultural influences. We all know what's right and what's wrong, don't we? The ideology on the one hand of "all amplifiers sound alike" is just as reasonable as the ideology of an audiophile who says that "all amplifiers sound different." You expect every set of opinions to have a dialectical opposite. The clashing of opinions is therefore perfectly reasonable, but for me to say that "I don't have any biases" is totally untrue. Scientists are not objective. One of the big myths about scientists is that they deal objectively with reams of data. Well, even if you discount cases of conscious fraud, you can look at many, many cases of science where people have fudged data, they've thrown out certain pieces of data that don't quite fit their preconceived notions of what the answer should be. Knowing what the answer should be influences everyone. I don't think that you can escape this. It's a human condition.

Low: A lot can be done to overcome it. If you were introduced to a new speaker behind an acoustically transparent gauze, you would have no chance beforehand to know what kind of tweeter is in there. It wouldn't hinder your ability to form an opinion on the product. Afterwards, it would be very relevant to say "Ah! Now I see that tweeter. Yes, that's what it sounds like when I've had previous exposure to it." The hard part is that, since you don't have that gauze and you do know something about the product beforehand, you mustn't mentally prejudge, mustn't let that get in the way.

Atkinson: But good reviewers—at least the ones who write for Stereophile—do put their biases to one side in order to be receptive to what their ears are telling them.

Low: Professionalism requires learning how to overcome your biases. In a demonstration, any good salesman knows that whatever you want to sell you play second. If you do an AB, B will sound better because you notice things the second time that you haven't noticed the first. But if you know this, then you ABA yourself. You don't let yourself fall into that trap. You do everything you can to recognize your biases and work around them.

Holt: What is the difference between so-called bias and previous acquired knowledge?

Bill Sommerwerck: There's a difference between having preconceived ideas due to experience and solidifying them into a fact. There's a difference between somebody saying "I've heard three speakers that use bass-reflex loading and they all sounded muddy" and solidifying that into "All speakers which feature bass-reflex loading sound muddy."

Low: It is a natural tendency for people do that...This tendency to draw conclusions partly relates to a review of my product where—sorry, Dick—one cable was preferred with a particular speaker, another cable was preferred with another speaker. Based on what to me was a grossly unscientific database of just two examples, a conclusion was then made in the review about one cable being more suitable with high-impedance speakers and one cable being more suitable with low-impedance speakers. To me, that was skydiving without a parachute. This happens a lot in little ways in many reviews. It's not an overall, overlooming problem, but it's endemic to our entire industry. We all practice it in little ways. After we've got a conclusion on a product, we look for the little handles that make it conform to previous experiences. Sometimes those handles are coincidence, they're not actually fact.

Lewis Lipnick: When you're a reviewer for a magazine—and since I've been reviewing I can see it in myself and I don't like it—you begin to lose touch with the reality of the way the consumer sees things. We're in an ivory tower, and unless we are on the consumer level and deal with them all the time and deal with the products the way the consumer does, we can take things for granted. We don't see that a slight comment we might make about a product, due perhaps to a preconception or a bias, will be blown way out of proportion by the readers. On the other hand, there's a lot of witchcraft in promoting high-end products, a lot of hocus-pocus.

Low: There's far too much of it. And far too many reviewers fall for it.

Arnis Balgalvis: In 10 years in the Audiophile Society, Larry Greenhill and I have met at least 110 manufacturers and dealers. And I have to say that when I look at what those manufacturers and dealers tell us, their bullshit is so much more overwhelming than anything that I've heard in this room.

Olsher: I'm really irritated when a manufacturer comes to me and says, "Look, I've got a wonderful product here. You've got to listen to it. It redefines the laws about how electrons flow down a cable." Now, speak about biases. Speak about hype. My feeling is "All right, I'll give you the benefit of the doubt. I don't care if the cables are made out of Swiss cheese, if it sounds good, I'll describe exactly what I heard." The fact that I can only try a cable with three or four loudspeakers should not be held over my head. You ask for my opinion, but then when you don't like that opinion you say, "Well, obviously you didn't try 20,000 different brands, therefore it's not an objective study and it's no good." You can't argue from both sides of the fence. You either get the full measure of a subjective type of review or you can go somewhere else.

Low: No, my complaint was not at all that not enough speakers were used. One speaker would have been adequate. What I'm saying is...that with the two speakers that were used, impedance wasn't even the only thing that distinguished those two products. I get calls to this day—this is the sense of the responsibility not appreciated—which go: "I have a high-impedance speaker," "I've got a low-impedance speaker: which is the correct cable for me?" Your point was taken extremely seriously by the reading public. Yet I feel it was not based on any actual opinion that you had.

Olsher: That wasn't a major conclusion. My major conclusion was to be very careful and match your cables to a particular system.

Low: Well, again you went beyond that and drew a specific conclusion.

Atkinson: The reviewer is forced into a situation where he has to generalize from the specific.

Low: But this is what Lewis said. You can't just ruminate about a theory and throw it out there to 60,000 people and not expect them to pay attention to it.

Sommerwerck: Consumer Reports publishes ratings of cars which include various breakdowns of breakdowns. And occasionally you'll see "Insufficient data to form conclusion." That is, they didn't have enough responses for a particular model of car for them to feel that they had a statistically reliable sampling. In general, therefore, a reviewer has the responsibility to ask himself, "Do I have enough data to reliably draw a general conclusion?" If the reviewer doesn't feel that he has enough data, he shouldn't draw the general conclusion regardless of what the specific facts are that he still presents to the reader.

Olsher: That's not quite the case when you're reviewing a single sample. You have to assume that the sample is typical of the manufacturing process, that it represents the population very accurately. We don't go out and buy five samples. We don't go out and double-check each of our findings. You call it the way you see it. It's not science, it's our best guesstimate given limited resources. It's my opinion, ultimately.

Sommerwerck: The fact that it isn't science doesn't mean that anybody has the right to draw a conclusion which is not scientific.