A Babel, a Babble . . . Page 5
Holt: At shows I've had a number of people come up to me and say, "Why don't you people work together on each review, so you can come up with a consensus?" And I said, "Well, for one thing, it would probably take us six months to get a review done, and for another thing I don't think our readers want a consensus." And then the guy says, "I do."
Olsher: I want to get back to this myriad of opinions: I think the more opinions, the more insight. Different people being able to kick products around from different angles, in different systems, fleshes out what the products can and can't do. The fact that there's disagreement is healthy. I mean, you can go to the Pope of Seacliff—HP—or the Axeman of Bronx—Peter Aczel—or, I haven't come up with a good name for Moncrieff, you don't need it, it's absolute crap. It's one man's opinion in a very narrow context. If a number of opinions all converge to say "This is a great product in all kinds of environments," you have a clear winner. But if you have a difference of opinion, why, I think that's just as intriguing as agreement. Disagreement is what horse races are all about. It's very important to keep it that way.
Holt: Well, I don't think there's any question but what if we did do group reviews and could come to a consensus about things, a review would have a hell of a lot more credibility.
Norton: Awfully difficult to do.
Mitchell: There are practical ways to do it...
Atkinson: The problem is, though, when we have sent some components to other writers, we often found there was almost no diversity of opinion. The second or the third person to listen to the piece of equipment actually has nothing to add to the original review but has spent a lot of time listening...
Mitchell: I should think hearing speakers in a different room would be very illuminating...
Lipnick: ...it'd be nice to be able to send products around. You say people will basically agree on certain things, and I think you're right, but...
Atkinson: They will agree over the description of the sound; the only disagreement will be whether this flaw rules it out of court for any recommendation, or whether this flaw is trivial. However, they will agree that there is a flaw in that place, in that manner.
Lipnick: But John, we have something here like a bunch of physicians who are specialists. A patient's sick and he goes to see a physician, and gets farmed out to other people who are all specialists. They get together later and talk about it. Now true, I'm talking about a sick patient, but what we've got I think could be interesting. For instance, I'm a musician. Some of these guys are technical. I'm not an engineer and I don't know what they're talking about. But I can probably tell them some musical things that maybe they don't know about. So I think that in some cases—maybe not every case—it might be interesting to have people who are specialists in different fields comment on certain products. Because, let's face it, if you're not an engineer you can't talk about the engineering. If you're not a musician you can still talk about music, but not in the same way a musician would talk about it. But that's the whole idea of having people who are diverse, I thought.
Atkinson: How many products are there that are worth that kind of intensive activity?
Lipnick: Not too many, but there are some.
Holt: I would say anything really expensive that gets a rave review, would warrant that.
Lipnick: Or one that got dumped on by someone.
Mitchell: Like the big Altec speakers that sounded lousy in Larry's room but might work in another room.
Norton: I think we shouldn't only think about the possibility of sending equipment around, but simply send the review around for follow-up comments from anybody else who happens to have heard the same piece of equipment.
Atkinson: I do think that any follow-up comments must appear after the appearance of the original review, otherwise the logistics would bog down the production of the magazine.
Holt: That's what I told that guy, that it would take six months to get a review out.
Greenhill: I wonder if that would make a reviewer hesitant to go to either extreme if he felt that his review was going to be second-guessed...
Deutsch: I remember the comment you had from a reader saying that the editor putting in a footnote, a comment about the sound of a product, is undercutting the reviewer, making the reader confused about what it really sounds like. If you include comments that differ significantly from what the primary reviewer found...
Atkinson: Except my experience with all of you is that you actually all do hear the same things. The argument against subjective reviewing, that it's just one man's opinion and there's no guarantee that it will concur with anyone else's, I do not think is true for this group of people, because you've all spent many years learning how to listen, and learning how to describe what you hear.
I think this is a trivial criticism of subjective reviewing in general. Provided that the people doing the subjective reviewing have taken that care. And have that experience. And these people who write for Stereophile, I'm convinced, have and do.
Norton: To return to measurements, a point I'd like to make is that the measurements are going to be made here in Santa Fe before the component is sent out to the reviewer. In general, that's the logistics I imagine for it. I, for one, wouldn't mind seeing the measurements but I don't want to see them early in the subjective evaluation process.
Atkinson: When Bob Harley and I were discussing this with Larry, we were all agreed that the measurements cannot be made available to the reviewer while he's doing his critical listening.
Olsher: A case in point was the Lazarus H1-A amplifier. I mean, you wouldn't think that in this day and age a solid-state output stage would have an impedance of half an ohm. That's unusually high and you would expect to have a speaker/amplifier interaction that behooves you to try a number of different loads to try to get around that point. But if you don't know that before the fact—people think an amplifier is easy to review, just hook it in your system and listen. Well, that's not the whole story—if you don't try four or five loads, you don't know what's going on.