A Reviewing Life

It's been 30 years since I began work on my very first equipment report, of the Goldbug Brier moving-coil phono cartridge, for Hi-Fi News & Record Review. That review appeared in the British magazine's May 1983 issue; I have lost track of how many equipment reviews I've written since then, but my review of the Vandersteen Treo loudspeaker in this issue is at least my 500th.

I touched on the subject of reviews in my Richard C. Heyser Memorial Lecture, "Where Did the Negative Frequencies Go?," which I was invited to give on October 21, 2011, at the 131st Audio Engineering Society Convention, in New York City. I had mentioned in that lecture that my education in electronics and audio was based exclusively on tubes. Even the logic circuits I constructed at school used tubes! But there was one experience that foreshadowed my career as an audio reviewer. For one of my final exams for my bachelor's degree, I was handed a black box with two terminals, and had to spend an afternoon determining what it was. (If I recall correctly, it was a Zener diode in series with a resistor.) That experience is echoed every time I try to characterize the performance of the audio components I review. Every product—be it a loudspeaker, amplifier, or CD player—is fundamentally a black box with input and output terminals. As a reviewer, I have to answer the question "What does it do?"

The next question, however, is more complex: "How well does it do it?" Throughout 2012 I appeared on panels at audio shows with other reviewers and editors, and the question came up more than once: How do reviewers assess the quality of the products they write about? One answer was also given more than once: "I compare the sound of the product against that of live, unamplified music."

Really? I know this is a common meme, coined by Harry Pearson back in 1973 when he founded The Abso!ute Sound. But, as I have argued in the past, it doesn't hold together with two-channel reproduction, in which the ambient sound at the original event is folded into the front channels. And as I demonstrated in my presentations last October at the 2012 Rocky Mountain Audio Fest, it is based on a fallacious assumption: that an accurate representation of the original recorded event is encoded in the grooves, pits, or bits.

But for the sake of argument, let's assume that a state-of-the-art audio component could make a system's sound indistinguishable from the real thing. With anything other than assaults on the state of the audio art, using the "absolute sound" to judge an audio component's quality is akin to trying to measure something with the meter set to the wrong scale. As every product falls short of meeting that goal, many by a large degree, the needle on the quality meter hardly moves from its rest position.

That doesn't even mean that such products are bad. Far from it. With very few exceptions, audio products are designed to a price point, which limits, often drastically, the resources with which a designer can work. To stay within the build budget, the designer must trade some aspects of performance off against others. So rather than proclaim, for example, that "this speaker has no bass" or "the midrange is colored" compared with "the absolute sound of live, unamplified music," the responsible reviewer must try to get into the designer's head. Why did the designer do it this way? Why does this speaker have no bass? Why is it colored in the midrange? What is the trade-off for which the designer has felt it worthwhile to sacrifice low-frequency extension or midrange neutrality? And why was that trade-off necessary?

Before passing absolute judgment on a product, therefore, the reviewer must first see how well the product measures up against what appear to be the designer's own goals. Then and only then can the reviewer assess how well the product performs, both against a more universal set of goals, and against a different set of compromises embodied in a competing product.

Of course, there is always what my erstwhile colleague Tom Norton, now technical editor at Home Theater magazine, used to call "designer tunnel vision": a concentration so narrowly focused on maximizing one aspect of performance that shortfalls elsewhere are apparently overlooked or unheard. But the reviewer can't take for granted that the designer is unaware of the problems—they may just be inevitable. Or, more likely, the problems don't intrude so much with the particular music the designer likes to play. This is why Stereophile's reviewers list all the recordings they use to form their value judgments. The review is analogous to inviting the reader 'round to listen to your system, and the choice of what music to play is critical. Very often, a product inserted into a system affects that choice, and that metadata becomes relevant information to readers of the review.

That's why the question reviewers are continually asked—"What's the best speaker at [insert price point]?"—is so difficult to answer. The most honest and least satisfying answer is "It depends." There is no "best speaker" that will suit every listener's needs. The reviewer lays out how the product sounds to him or her, but each reader then must examine her or his own preferences. Do I need maximum low-bass extension? Do I desire midrange accuracy? Do I value maximum loudness? Do I want holographic imaging? Can I give up one of these to get maximum performance elsewhere?

Most important, both reviewer and reader need to answer this binary question: "Do you like the sound of this component, yes or no?" The Golden Rule for all listeners: To thine own ears be true.

dalethorn's picture

I'll never forget what Henry Kloss used to say: "The shortest distance between two points is a straight line, or the line that's straightest under the circumstances."

pwf2739's picture

Mr. Atkinson,

I would like to congratulate you on your many years of audio reviews and your dedication to our industry. It is people like you, Harry Pearson (even though you may slightly disagree with his methods), J. Gordon Holt and Julian Hirsch who have set the benchmark for all of the audio reviewers and experts who guide today's audio aficionado. Our's is an industry that has become increasingly difficult to make an informed decision. It has become quite challenging to be able to demo a component in which one may be interested. Therefore, audio reviews have become, if not a starting point, but the guiding principal by which new product is purchased. The body of work you and your colleagues have published, and continue to publish, are looked upon as an absolute truth. I hope your work and wisdom in helping the audio hobbyist continues for years. 

MVBC's picture

"Before passing absolute judgment on a product, therefore, the reviewer must first see how well the product measures up against what appear to be the designer's own goals. Then and only then can the reviewer assess how well the product performs..."

Yes indeed, these principles allow an almost infinite range of shades that will entice readers and please manufacturers... What's not to love, hey?devil

John Atkinson's picture

Yes indeed, these principles allow an almost infinite range of shades that will entice readers and please manufacturers... What's not to love, hey?devil

With all due respect, you're an idiot.

John Atkinson

Editor, Stereophile

MVBC's picture

Your excessive, childish reaction proved my point beyond any doubt.smiley

sommovigo's picture

With regard to any aesthetic pursuit there seems to be no guidance other than "that which blows up one's own skirt," as it were, and so it seems that the job of the reviewer would be only to communicate attributes as well as possible without opinons regarding absolute quality - unless, of course, there is a matter pertaining to an overt defect in workmanship or flaw in design (in which case the reviewer would be seeking to protect the reader from potential damage, for instance).

What is the overall object of our hobby, if not the accurate reproduction of music? Well, perhaps it is not that at all - despite the best efforts of philosophers to superimpose such ethics onto the enthusiasm we share. I suspect this may largely be due to the unnecessary narrowing of the definition of "accurate" as it pertains to the phrase and concept of "reproduction of music" - because "music" itself is something of a vague abstraction that depends as much on the listener as it does on the producer. "Sound" is no better, of course, requiring a Schrödinger to observe the cat ... that is to say: "music" and "sound" are perceptions, the result of an observation collapsing a wave-function. (I think that the most we can say, regarding that koan of the tree in the forest with no one there to witness it, is that it would seem that the conditions for the perception of "sound" would be present under the proposed circumstance, but without an observer there could be no observation ... and because "sound" is an observation and based upon a stimulus, the answer is probably No).

"Music" is, therefore, what you think it is. Growing up in and around what you might call the American Punk Era, I might think that Sex Pistols, Black Flag, Bad Brains, Circle Jerks, Suicidal Tendencies, Dead Kennedys, Butthole Surfers, Fear, Social Distortion, Buzzcocks, etc qualify as "music" ... and I assure you, my parents did not agree. Meanwhile, I was equally at odds with my mother's love for Percy Faith and Johnny Mathis, and the whole family agreed upon the non-music status of Lawrence Welk. Nevertheless, our neighbors - the Heebners - found the dulcet tones Herr Welk and his accordian to be divinely inspired.

The function of a reviewer seems best executed, in my opinion, when they are able to use language in a way that communicates the essential characteristics of the Device Under Test while refraining from judgement - one man's poison may be another man's perfume, and so it becomes somewhat incumbent upon the reviewer to avoid potentially spoiling something for a reader that might, in fact, find aggressively-balanced midrange and incoherent soundstaging the veritable bees-knees. I think that it is a mistake, therefore, to impose or superimpose a value system onto an aesthetic pursuit.

And this is the trouble I find with approaches to judgement as we've considered them, philosophically, over these many years. Carrying a lance and standard forces a choice among sides, of which we've had only three from what I've been able to gather over these last two decades (covering the period of my active avoidance of a real job):

  1. The "live music" camp that insists the sole purpose of a home audio system is to create the illusion of a live musical event (subjectively) in the listener's home.
  2. The "microphone feed" camp that insists the sole purpose of a home audio system is to re-create the microphone feed as accurately (objectively) as possible.
  3. The "Hedonist" camp that insists the only thing that matters is that the individual listener gain the maximum amount of pleasure from their system, no matter what the other two camps might think of it.

Of course, in the Grande Venn Diagram it is possible to overlap any and all of these, as they aren't necessarily mutually exclusive, so it is rather a matter of emphasis or the importance or meaningfulness of emphasis that should demand some consideration.

I suspect that the overarching question that needs answering is as simple as "Does it bring the listener to Eargasm?" And since this is the kind of question that depends entirely on the individual listener, their perceptions and values and delights and buttons, it seems to become an impossible task for the reviewer to proclaim an aesthetic value-judgement that would serve anyone other than themselves.

When it comes to Aesthetics, it may be that a reviewer serves best who opines least.

Ariel Bitran's picture

thank you for this post. 

walshbouchard's picture

"With all due respect, you're an idiot."

John Atkinson

Editor, Stereophile


Lol proves his human  and not just a HIFI reviewer , and anyway you were being a bit cynical, if you had said that to Ken Kessler   he would have called you far worse