Think Pieces

"If it sounds good..."

John Atkinson  |  Apr 26, 2012  |  First Published: Dec 26, 1992

"...and measures bad, then you're measuring the wrong thing!" If one motto could sum up this magazine's philosophy, this would be it. Too many times we have discovered components that sounded musically fabulous while offering measured performance that was, at best, merely competent. Yet recently, I'm starting to lose confidence in that old saw.

On a number of occasions, I or another of Stereophile's reviewing team has heard a product sounding flawed in ways later revealed by measurements. A closed story, you might think—but consider the NEAR-50M loudspeaker reviewed by Dick Olsher in this issue. Despite hearing many good things in the speaker's sound, Dick was bothered by a tonal-balance problem in the low treble. He was also disturbed by a lack of integration between the tweeter and midrange unit. When I measured the '50M, my response graphs (footnote 1) pretty much explained why Dick heard what he heard. Nevertheless, other reviews of this loudspeaker have been ecstatic in their praise, one even stating that it was "one of the most transparent and balanced dynamic loudspeakers available at any price" (my italics).

The question begs to be asked: Are these other reviewers hearing what Dick and I heard? The answer must be yes. Putting to one side the problem that language appears to be an inefficient medium for conveying ideas, without an assumption that experiences can be shared by diverse listeners, there is no philosophical basis to publish any review magazine.

So why are some listeners bothered by what, to them, are gross flaws, while others are so enamored of other aspects of reproduced sound that they overlook entirely what the first listeners found so disturbing? The difference must lie in the calibration of their ears.

I have written in the past that the most reliable indicator of quality in hi-fi components is the listener's holistic reaction to the sound (footnote 2). But on what bedrock is this reaction founded? "Observation requires of the observer a considerable degree of interpretation based on expectations and already-formed models and structures," I wrote earlier this year (footnote 3).

For the listener's reaction to sound to truly reflect the performance of a component, therefore, he or she must have suitably informed his or her subconscious by becoming a) familiar with the range of performance available at any price—otherwise this phrase degenerates into empty word-spinning—and b) as familiar as possible with the real sounds of real musical instruments in real acoustic spaces.

The latter is an essential part of an audiophile's education: If you don't sensitize yourself to the differences between, for examples, the tonal quality of an oboe and that of a soprano saxophone, or the difference between a Fender Stratocaster electric guitar or a Gibson Les Paul, then you won't detect the fact that a loudspeaker can confuse their sounds—something that J. Gordon Holt touches on in a 1964 essay reprinted in this month's "30 Years Ago" feature.

If audiophiles familiarize themselves with the attributes of live sound, they can then grade changes in sound quality accordingly. (I've found this to be true, paradoxically, even if they then use artificial, multi-miked, multi-tracked recordings.) If they don't do this, however, then their value judgments will be as topsy-turvy as in the case of the reviews in other magazines of some of the components we review this month.

To return to the head of this little diatribe, therefore, when you read that a component "sounds good but measures bad," you must examine what basis the writer has for determining the meaning of the word "good." To sound good, a product must at least offer competent engineering, I feel, and it may turn out that "sounding good" does not necessarily mean the same to some listeners as "neutral" or "accurate." My role as editor of this magazine is to ensure that, for Stereophile writers at least, it always does.—John Atkinson

Footnote 1: The graph above is not the NEAR speaker. But it, too, is an example of a loudspeaker with terrible measured behavior that was proclaimed by another magazine editor as a new reference at its price.

Footnote 2: "The Puzzle of Perception," Vol.15 No.2, February 1992, p.7.

Footnote 3: See A Second Way of Knowing: The Riddle of Human Perception, by Edmund Blair Bolles, Prentice Hall Press, 1991; and Consciousness Explained, by Daniel C. Dennett, Little, Brown, & Co., 1991.

COMMENTS
Duck851's picture
Submitted by Duck851 on April 26, 2012 - 11:47am

I think it would help me understand the tests and results a lot better if I could watch a video of the speakers (and components) being tested.

andy_c's picture
Submitted by andy_c on May 1, 2012 - 7:01pm

Another good source of information about loudspeaker testing can be found in the manual of the excellent freeware for this purpose called Room EQ Wizard. This PDF file can be found here:
http://www.hometheatershack.com/roomeq/REWV5_help.pdf

John Atkinson's picture
Submitted by John Atkinson on April 26, 2012 - 12:05pm

Quote:
I think it would help me understand the tests and results a lot better if I could watch a video of the speakers (and components) being tested.

I am giving an hour-long seminar at the Newport Beach show in June on what measurements like this mean. I hope to make the video of that seminar available on this website.

John Atkinson

Editor, Stereophile

RobertSlavin's picture
Submitted by RobertSlavin on April 26, 2012 - 1:43pm

The book The Complete Guide to High-End Audio by Robert Harley explains the measurements in Stereophile. Harley was the Consulting Technical Editor of Stereophile back in the 1990s.

Robert

John Atkinson's picture
Submitted by John Atkinson on April 26, 2012 - 2:38pm

Quote:
The book The Complete Guide to High-End Audio by Robert Harley explains the measurements in Stereophile.

Robert Harley never performed speaker measurements for Stereophile. I developed the speaker measurement regime featured in Stereophile and did the measurements for the speakers he reviewed for us. The section on speaker measurements in Harley's book used graphs that I had prepared for him. You can find articles explaining my speaker measurements at www.stereophile.com/reference/99/index.html

www.stereophile.com/reference/100/index.html and

www.stereophile.com/reference/103/index.html.

John Atkinson

Editor, Stereophile

John Atkinson's picture
Submitted by John Atkinson on April 27, 2012 - 12:07pm

You can find our review of the first edition at www.stereophile.com/reference/102/index.html.

John Atkinson
Editor, Stereophile
remlab's picture
Submitted by remlab on April 26, 2012 - 3:09pm

I think I remember reading somewhere that Robert Harley's entrance into high end audio reviewing was through a stereophile essay contest. Is that true or false?

John Atkinson's picture
Submitted by John Atkinson on April 26, 2012 - 3:20pm

Quote:
I think I remember reading somewhere that Robert Harley's entrance into high end audio reviewing was through a stereophile essay contest. Is that true or false?

Mostly true. I advertised for a full-time technical editor for Stereophile in the spring of 1989. The top 3 candidates were Robert Harley, Guy Lemcoe, and Barry Willis and to choose between them, I asked each to write a review. I hired Robert, but Guy Lemcoe joined our team of reviewers and Barry Willis wrote news items and articlea for Stereophile, as well as a few reviews, for many years. Guy is now retired to Florida but Barry is still around, contributing a column to UK magazine Hi-Fi News.

Harley resigned from Stereophile in June 1997 to join the late and unlamented magazine Fi.

John Atkinson

Editor, Stereophile

remlab's picture
Submitted by remlab on April 26, 2012 - 3:41pm

 Electronic components don't interact with the listening room. Loudspeakers do. This is why published loudspeaker measurements are so critically important to the potential buyer.. 

remlab's picture
Submitted by remlab on April 26, 2012 - 5:09pm

"In fact, Mr. Rothman declares that the conventional wisdom that the timbral differences between different types of instruments lies in the spectra of their harmonics is wrong. Certainly a correlation can be drawn between the fact that their sounds are different and that their harmonic spectra recorded in an anechoic chamber are different. Rothman states, however, that "the spectra of a note on the oboe sampled at two different points in an auditorium differ more than the same note played on an oboe and a trumpet." Yet the subjective difference between "oboe-ness" and "trumpet-ness"---the identities of their acoustic models, if you will---is vast."

Wow!

floydianpsyche's picture
Submitted by floydianpsyche on April 27, 2012 - 8:08pm

Would there be a time in the future, where they measure your ear and match a speaker for that? ? - Pradeep

mrplankton2u's picture
Submitted by mrplankton2u on April 28, 2012 - 4:28pm

John deserves a lot of credit and thanks for spearheading the effort to bring about a more objective and accurate method of measuring loudspeakers. He has continually striven to establish connections between what the reviewers have heard and specific features found in the measurements. On that score, he and Stereophile clearly stand alone in the review industry and both should be commended for it. If I had one suggestion, it would be to conduct measurements in a standardized listening space (like a 12X20 room with 7 foot 6 inch ceilings, hardwood floors, and a large area rug in the middle) and take measuremnts from the same listening location at sitting height for every speaker. The spliced bass measurements and the on/off axis tweeter measurements are great but what I want to know is - how does the speaker compare to the last one that was measured from the same distance with the same toe-in position in the exact same listening location. I don't care if it's a ribbon, book shelf, floor stander, point source, electrostatic, line source - whatever. It's the manufacturer's job to build a speaker that will sound convincing in a "typical" living space under "typical" listening conditions without expecting the listener to adjust the listening space or location to suit the foibles of a particular loudspeaker design. I would give a lot more weight to measurements taken from a standardized listening position in a standardized room than those that are spliced or are taken from locations that tend to suit a particular design layout. With different gated measurements from a typical fixed listening location, we should be better positioned to compare how one loudspeaker will interact with a listening space versus another and that for me is a very important criterion for evaluating a loudspeaker. - My $.02...

John Atkinson's picture
Submitted by John Atkinson on May 2, 2012 - 9:14am

Quote:
If I had one suggestion, it would be to conduct measurements in a standardized listening space (like a 12X20 room with 7 foot 6 inch ceilings, hardwood floors, and a large area rug in the middle) and take measurements from the same listening location at sitting height for every speaker.

We used to have a standard listening room 1991-1998, when the magazine was based in Santa Fe, but real estate costs prevented that when we moved to New York in 2000. So each reviewer auditions speakers in his own room. However, I always publish a spatially averaged in-room response in my own speaker reviews, always performed at the listening position in the same room in the same manner -- see for example, fig.7 at www.stereophile.com/content/sonus-faber-amati-futura-loudspeaker-measurements, which compares 3 speakers measured under identical conditions. I also do the same for the speakers that Michael Fremer reviews, in his room, and for those reviewed by Wes Phillips, when he was an active reviewer.

John Atkinson
Editor, Stereophile
remlab's picture
Submitted by remlab on May 1, 2012 - 4:45am

The CNRC has a "typical listening room" to complement their anechoic room... Wouldn't be a bad idea!

Doctor Fine's picture
Submitted by Doctor Fine on May 17, 2012 - 12:38pm

The frequency response curve measurements you take in your test rooms of the actual in-room response are as close to priceless information as one can get out of a review, John. I all ways look for these first in any of Stereophile's reviews and am distraught in those rare instances where they were not included in the article.

What happens most of the time is that this one graph of how it worked in that room is amazingly informative about what the speaker is doing.  Or will do.  Perhaps in a lot of rooms.   Good bad or indifferent.

A terrible looking response curve is a real good clue something is only half baked.  And in that sense the curve is a lot like listening to the thing itself.  It is like, you know, Beethoven was deaf when he composed his last works.  He only looked at the graphs, ha ha.

X