"When things go wrong...it hurts me too"
As a writer and editor, I was relatively quick to appreciate the advantages offered by word processors over typewriters, and have built up quite a body of experience over the last nine years with a variety of computers, disc drives, printers, and modems. With the exception of finger problems and software bugs—which in themselves can drive a man to strong liquor—I have had no reliability problems with any of that hardware despite constant use and, with the portables I use, a considerable degree of travel abuse. Yet when I look back over the magazine's experience with components sent for review, high-end components appear to have an appalling track record in comparison with computer hardware, despite their similarly complicated electronic nature.
For example, in just the last 12 months or so, we have had (footnote 1):
An amplifier supplied for review with its output transistors wrongly wired. Although it appeared to be working correctly, the sound was disappointingly bad. Only after the review sample was returned to the manufacturer did the reason for its poor performance become apparent.
A tube power amplifier that broke while driving Apogee Caliper speakers, which are not too extreme a load, in my opinion. When fixed by the manufacturer, the same amplifier featured a significant level of hum which couldn't be eradicated.
A manufacturer who had to supply us with three samples of a preamplifier before we had one that worked correctly.
A tube preamplifier that arrived dead in the box. The culprit was a blown fuse—before it was plugged into a Stereophile wall socket.
A power amplifier that could not be auditioned at all due to the fact that the circuit breaker in the house wiring cut out every time it was turned on.
A very expensive hybrid power amplifier, of which four samples failed in a row (after the favorable review was written, of course).
Another very expensive hybrid power amplifier that had persistent tube failure.
A tube preamplifier that went unstable at low frequencies, leading to the destruction of a power amplifier.
More than a few CD players based on Magnavox chassis that had faulty or intermittently working transports.
A portable CD player with a laser that refused to follow the pit spiral.
One CD player that had its de-emphasis switched in all the time whether the discs needed it or not.
A digital decoder that underwent a series of modifications during the review process to remove a weakness concerning the pickup of RF interference.
A handful of box loudspeakers that arrived with only one of the pair in working order. One model had one of the pair's drive-units wired out of phase with the other.
Four box loudspeakers that self-destructed during not-very-rigorous testing.
A pair of dynamic loudspeakers from a complete production batch that featured woofers totally out of specification, resulting in a redesign during the review period.
An electrostatic loudspeaker that destroyed two of the amplifiers with which it was used.
A speaker manufacturer who recalled a review pair of loudspeakers before we had a chance to listen to them on the grounds that he had updated the design. The replacement pair were likewise immediately recalled and replaced with a third pair. Of this pair, one speaker was dead out of the box and the drive-unit phasing appeared to be different in each speaker.
An expensive high-end loudspeaker whose manufacturer apparently had considerable problems in supplying two that sounded alike. A knob then fell off the control unit and the terminal posts worked themselves loose.
An active loudspeaker whose bass-amplifier sensitivity was set so high that, even with its maximum attenuation, the level was 6–12dB too high compared with the midband.
A subwoofer that had its input and output crossover connections reversed, resulting in a response, when wired up "correctly," that didn't extend any lower in the bass than a BBC LS3/5a.
Another subwoofer which also extended no lower in frequency than a small monitor when measured, whereupon the manufacturer asked for the review to be killed on the grounds that he hadn't supplied it specifically for review, and in any case, the subwoofer would only work correctly with his particular loudspeakers.
A tonearm whose distributor couldn't supply a working sample for a significant length of time.
A turntable with a suspended subchassis for which there was a complete lack of vibration isolation. The manufacturer "solved" this problem in subsequent production by removing the suspension altogether.
A turntable which, as supplied by the distributor, barely worked and was missing a significant amount of its accessories.
A moving-coil cartridge that arrived loose in its packaging and minus its stylus.
And in this issue's equipment reports, you will note from Martin Colloms's review of the Klyne SK-6 preamplifier that the manufacturer replaced the review sample while the review was underway. Though the SK-6 has been in production for a year or so, the first sample that we received for review was, in fact, founded upon a revised circuit board. Some time into the review process, Stan Klyne found out that the entire batch from which the review sample had been drawn sounded considerably less good than the initial production, the culprit turning out to be the revised board. He decided that they should revert to the older board and scrapped all of the bad batch, the only one to get away being Martin's review sample.
Stan accordingly asked if it were possible to submit a second sample, typical both of current production and of the generation already in consumers' homes. We had no objection, and Martin finished the review, repeating the line-stage measurements (the phono board was the same in both samples) and the auditioning.
As you will see, the review findings based on the second sample were generally favorable, although Martin did have reservations about the phono stage's overload margin and linearity. However, while Stan Klyne was not pleased with his findings in this area—see "Manufacturers' Comments" in this issue—they were prepared to accept them. They were, however, extremely disturbed to find out that the review as printed would contain a significant amount of negative comment concerning the first, defective sample. As current production of the SK-6 is identical to the second, better-sounding sample, they felt that to include any mention of the first would misinform or confuse Stereophile's readers: "The review is overly negative, frequently contradictory, and confusing because it refers to the first unit for too much of the time," stated Klyne's Janice Arnold in a letter following Stereophile's sending Klyne a preliminary copy of the review.
Well, I'm afraid that them's the breaks. As I stated in "As We See It" in December 1988, if it turns out that a review sample was faulty, we will gladly take delivery of a second, third, or even a fourth sample, but the performance of all the samples will be reported in the body of the review. To quote from December's column, "the writers are instructed to include in their reviews all their experience with all the samples they've received, not just the most recent or best-functioning."
In a sense, everything that happens during the course of a review is "on the record." The reasoning behind our taking such a hard line is as follows: If it is possible for a Stereophile reviewer to receive a faulty or unrepresentative sample of a component, with all that would be at stake were the product to receive a negative review, then it is probably more likely for one of the magazine's readers to do so. The magazine's primary responsibility is to its readers. Ergo, quality-control problems must be reported in the review, and to make exceptions for some companies would be both inconsistent and unfair.
Against this, it could be argued that a reviewer is more likely to receive a defective sample than a reader, he or she often experiencing a sample from the first production run. After all, doesn't every magazine want the very first review sample of any product? In addition, the dealer is there to act as a buffer between a manufacturer's lack of QC and the consumer: if a company sends out products to its dealers that turn out to be defective, it gets them straight back. This is the so-called "Beta" testing where the manufacturer relies on third-party experience to reveal problems that didn't show up in the in-house, "Alpha" testing.
I am afraid, though, that I have little sympathy with these arguments. In my opinion, a magazine reviewer's listening room is a singularly inappropriate Beta-test site. The reviewer represents the interests of his or her readers, not those of the manufacturers. Not to inform the readers of bugs and failures, perhaps substituting a private word with the appropriate manufacturer, borders on behavior nearer to that of a consultant than a reviewer.
It is also relevant to a review's findings whether or not the manufacturer can make their products to a consistent standard. It was either Laurie Fincham of KEF or the late Spencer Hughes of Spendor—I am afraid that I can't remember whom—who succinctly defined the skills required of a manufacturer: first, to be able to design a worthy product; second, to be able to make it consistently and reliably. Each is as important as the other, and both are relevant to the consumer, not just the first.
The other aspect of reliability concerns when a product fails in the consumer's hands. VTL's David Manley, in responding to a letter in this issue from a Mr. Belterri complaining about the supposedly poor treatment he has received following the failure of a VTL product, asks what exactly is Stereophile's stance concerning publication of letters from readers who have had problems?
We actually receive only a small number of letters of complaint and take each one at face value. Whether I choose to publish or not depends on a number of factors, principal among which is whether we have received other letters complaining about the same manufacturer. I always allow the company complained about the right of reply. In this case, Mr. Belterri and Mr. Manley can't both be correct in their published statements, but at least it appears that Mr. Belterri did ultimately receive his money back and VTL did find another customer for the disputed amplifiers.
The moral to be drawn from this unfortunate exchange of letters, however, is that when a company attempts to prove to one of its customers that he or she is in the wrong, winning that argument will always cause it to lose in the broader scheme of things. Should a company act as though the customer is always right? It would be wise to do so, as far as I can see.
Footnote 1: It would be unfair to name names, as all the manufacturers concerned have already had their problems aired in print. A careful reading of the last 14 or 15 issues of the magazine, however, will reveal who the featured manufacturers are.