Subjective Loudspeaker Testing

One of the most firmly-established audio platitudes is the one which says "The specs don't tell the whole story." One reason for this, of course, is the fact that most manufacturers, preferring to sell their products on the basis of emotional appeals in ads rather than on hard, cold performance claims, do not attempt to make their specs tell the whole story. They publish only the most basic, and the most impressive-looking of their specs, although they are no doubt fully aware that what they do publish is only a very small part of what even the most astute buyer would need to know in order to predict what the component will actually sound like.

Things have come a long way, though, since the days when designers would expect two amplifiers that measure the same on IM and response tests to sound alike. Objective tests have become far more sophisticated, and meaningful, than they were even five years ago, and have reached the point where, at least for some components, measurements do tell just about the whole story.

Recent research on pickups, tuners, and tape recorders have turned up many new tests for these which allow the designers to predict with a fair degree of accuracy how they will actually sound. But in the realm of loudspeaker testing, we are still pretty much in the dark ages.

While it is true that a loudspeaker which measures poorly will almost invariably sound poor, it is equally true that speakers that measure exceedingly well often sound almost as lousy, in a purely subjective listening test, as ones that are objectively poor. This is why, of all components, the loudspeaker is the one that must be selected mainly on the basis of how it sounds. And it is why The Stereophile rarely bothers to confound its readers, or itself, with detailed measurements of the loudspeakers it reports on.

One amplifier manufacturer, in his Comment on one of our equipment reports (the Dynaco Stereo 120, in this issue), indicated his doubts about the reproducibility of subjective evaluations. The point is well taken, for a number of uncontrollable and largely unpredictable factors all conspire to affect the way a given loudspeaker will sound to a given listener. Room acoustics, amplifier damping, speaker placement, personal "preferences" and variations in program material will all determine how a listener will react to a loudspeaker, so it is difficult to see how a subjective report, of the kind we favor in The Stereophile, could possibly be of value to the cross-section of highly critical listeners who subscribe to The Stereophile.

To some extent, such reports must be "interpreted" by readers, much as they learn to interpret record reviews—in terms of whether they usually agree or disagree with the reviewer. The fact that a very large percentage of Stereophile readers find that they do agree with most of our equipment evaluations is not however a result of dumb luck, or of gentle persuasion. It is the result of a carefully developed system for subjective testing, that has been evolved over a period of 18 years of highly educational mistakes in the selection and use of all kinds of components.

Minimizing the Variables
We'll examine in detail The Stereophile's techniques for minimizing the variables in loudspeaker performance and providing test setups that actually reflect the loudspeaker's inherent performance capabilities. One of our readers, who asked to remain anonymous, related to us the results of an interesting series of tests he saw conducted on one of our top-rated loudspeaker systems. The response checks showed that the system had virtually no deep bass, a mid-bass peak, a midrange slump and a high-end rise. Further checks showed gross distortion at input levels over about 6 watts, and a definitely limited (although adequate for row-H listening) maximum output-level capability (footnote 1). Said reader then went on to ask how we could possibly consider such a loudspeaker to be one of the best available.

Well, we weren't going to argue with measurements. All we did, in reply, was reiterate what we had found on the basis of our listening tests: that the system in question sounds smoother and cleaner than any other we've heard to date, it reproduces without apparent attenuation (and very cleanly) bass notes that, according to the musical scores, are well below the limit indicated by the reader's response tests, and the system is as natural a reproducer as anything we've come across. And that, in the last analysis, is what counts.

A Matter of Taste
Now, this is the point in our argument where some of our more knowledgeable readers will leap from their seats and say "Aha! That's where they're making their first mistake. It may sound good to them, but different people hear differently, and what's real to The Stereophile may not be to somebody else."

Only partly true, we counter. Different people do perceive sounds differently, for the simple reason that different people listen to different aspects of a total sound.

Suppose we take two listeners, whom we shall call A and B, to a live performance of the Eroica symphony. If they sit side by side, both listeners will be exposed to essentially the same set of sound waves, but listener A may concentrate on the pattern of harmonies and instrumental textures, while listener B may be "tuned in" to the main themes and the rhythmic groupings. Each is hearing the entire complex of sound, in the sense that his eardrums and nervous system are responding to it, but each is perceiving—that is, is consciously aware of—different aspects of the sound.


Footnote 1: You may guess what system was under test, but don't bother to write and ask us to confirm your guess. The reader did not wish the loudspeaker identified either.
ARTICLE CONTENTS

COMMENTS
Glotz's picture

lol.. Sorry, I'm a sucker for cheap jokes.

Poor Audiophile's picture

Sorry what?
Wait. Are you talking about the attractive young woman? It is an odd picture!

Graham Luke's picture

she's found HER sweet spot....Rrrrrraaow!

Ortofan's picture

... evaluated speakers by listening to them outdoors?

Poor Audiophile's picture

Great question!!

Bogolu Haranath's picture

When they want to evaluate outdoor weather-proof speakers :-) .......

johnnythunder's picture

they were specifically speakers made for outdoor use ? What is your point? What are you smoking? Why, after all the genius, brilliant points JGH makes do you arrive at the comment you made? What relevance does it have besides your own inherent continual naysaying and trolling on this site? Why don't you admit that you always knock a Stereophile reviewer's comments with some nonsensical and worthless post about a random measurement or the fact that the expensive component being reviewed is an automatic rip off compared to the budget component that you think measures better and therefore must SOUND better? Why do you read Stereophile if you continually disagree with the subjective based evaluation methods (brilliantly stated by JGH btw) that Stereophile uses to this day?

Ortofan's picture

... expensive versus "budget" components until you've read HR's review and seen JA1's measurements of the $1,200/pr Wharfedale Linton Heritage speakers. Then the question will be why can't other companies produce speakers that sound as good and measure as well for a similarly reasonable price?

johnnythunder's picture

My issue has been your knee jerk reaction without auditioning or listening to a component, that an "expensive" component that you don't feel measures perfectly or is somehow flawed in an engineering sense, is a rip off compared to something inexpensive that measures well. It's as simple as that. I don't believe in expensive meaning better by any stretch. I believe that we all have priorities in what we like about a component. I will listen and do a little research. I have purchased great sounding components that do not measure well and I don't care. I am also of the economic stature that the most I've spent on any one component is $2500. And I also want something to look nice too. You pay for more elevated product design and better materials for construction. SOmetimes that correlates to better sound sometimes just looking better and sophisticated.

Ortofan's picture

... are considered to sound great but do not measure well will explain why the two outcomes, for them, seem to be mutually exclusive.

johnnythunder's picture

The same components in a different designer's hands will sound different based on that "X factor" of little adjustments here and there that add up to something special on one hand or blah on the other hand. I work in a creative field - photography to be specific. You can give two photographers the same cameras and lighting diagrams and subjects and their results would be substantially different because one photographer has more charisma, soul, feeling and that stuff is more important to the finished product than the technical. It happens all the time. Countless times. Art - and I think the creation of many of the bespoke audiophile products we read about here count as ARTISTIC musical communication devices - is not that easy to make. Some of the designers of these products have more SOUL than others and care about different things in the creation of these components. Some WANT absolute clarity and some want the warmth of a Guarneri. To each their own. No measurement outside of ones that would designate a component as severely flawed and dangerous, should matter over a careful audition to find out if the component being considered speaks to you, makes you feel and hear the music more deeply.

Ortofan's picture

... whether or not you want your components to be imposing any amount of extra "clarity" or "warmth" upon all of the sound signals that pass through them, or do you want to hear the closest approximation to what the particular composer, performing artist(s), producer, and recording and mastering technicians intended for you to hear?

Bogolu Haranath's picture

Do you mean, you don't want any 'useful additive distortions (a.k.a. UAD)'? :-) .........

johnnythunder's picture

that I prefer a small amount of tube based warmth in my sounds. Many of my favorite engineered and reissued favorite recordings were recorded and remastered with analog equipment. I think "accurate and 100% truthful to the source" or "the engineers intentions" are bars that are nearly impossible to live up to at home with a fairly modestly based system. Too many variables and too many choices to make. Only at "Michael Fremer levels of system expenditures" do I feel you can get a system that is that perfect mix of detail and body ( and I think MF's preferences are towards extraction of detail and transient detail and clarity rather than warmth.) Nor do I really feel that I'm insulting anyone by not listening to something exactly precisely the way it was intended (I watch way too many classic movies on bluray to worry about that - the directors largely shot on 35mm or 65mm film and expected their films to be experienced in a theatre projected that way.) I'm not in favor of gross distortions (I haven't owned an amp with tone controls in 30 years) but just as I like my TV color w very rich black levels I prefer my audio sounds a little warm rather than lean. Poorly recorded pop or classical from the early days of digital - tinny and harsh and thin - sound much better TO ME with the added warmth I get from the tubes and other peripherals in my system. I'm not saying its correct or that everyone else needs to enjoy it that way. It is what I like and prefer and that's all that matters.

Ortofan's picture

... tube buffer stage devices from either iFi or Icon?
https://ifi-audio.com/products/micro-itube2/
https://iconaudio.com/all-valve-buffer-amplifier-ba3

Jack L's picture

........- tinny and harsh and thin -" quoted Johnnythunder.

Hi

Bingo. Can't agree more.

That's why I've switched to vinyl 4 years ago (1,000+ stereo LPs now) & never look back. That said, I still keep my digitals: CD, DVD-audio, blu-ray, DAC streaming etc on back burner, for convenient casual listening only.

Like MF, I go for fast, transparent, full detailed music like the best solidstate amp but with emotional engagement like the best tubes.

Since I could not find commercial amps with such 'hybrid' sonic qualities, not even the Audio Note Japan USDF125,000 power amps (being a little bit slow & a bit too much tube warmth), I had to design/build tube phono-pre-amp & tube power amps myself that give me the 'hybrid' sonics I want. Now I got it finally, for dirty cheap costs, thanks to my electrical/electronic engineering background.

Listening is believing

Jack L

Bogolu Haranath's picture

FWIW ..... JGH does mention about outdoor listening :-) ........

Bogolu Haranath's picture

Wonder whether Wilson Audio listened to their WAMM speakers outdoors and then moved them inside for final tuning? ...... That would qualify the WAMM speakers as "tower of science" :-) ........

jimtavegia's picture

will not work unless you intend to listen outside. You should evaluate everything where you intend to use it.

Ortofan's picture

... shouldn't ever be measured in an anechoic chamber.

jimtavegia's picture

I guess that kind of testing will work for you. The reality is that we want amps that measure flat and speakers that can do close to the same (although none do) so we can make adjustments in our rooms if we choose, but I doubt most people do not. This is why speakers are always the weakest link, especially their cabinet resonances. Matching speakers to room will always be the hardest thing to do.

Glotz's picture

She seems a bit off-kilter in her listening position... like she's either in pain holding that lp or she is quite... dead.

volvic's picture

That is one of my back exercises that I do while listening, only difference is that the record is on the turntable and not in my hands.

Bogolu Haranath's picture

She is listening to 'Love Me Tender' now ........ She wants to listen to 'Martha My Dear' next on the album in her hands :-) ........

Bogolu Haranath's picture

May be she is listening to in-ceiling speakers? ........ That console may be just playback gear? :-) .......

Bogolu Haranath's picture

Fast forward to 2020 ...... She could be listening to Bluetooth in-ear phones and texting on the smart-phone :-) ........

Bogolu Haranath's picture

Loudspeakers and their drivers design and engineering are far more advanced now than, they were in the 60s and 70s ....... Thiele-Small parameters were first published in the late 60s and early 70s for example :-) .......

ok's picture

is that they prompt you to read present articles as historical ones.

Glotz's picture

A Matter of Taste

Suppose we take two listeners, whom we shall call A and B, to a live performance of the Eroica symphony. If they sit side by side, both listeners will be exposed to essentially the same set of sound waves, but listener A may concentrate on the pattern of harmonies and instrumental textures, while listener B may be "tuned in" to the main themes and the rhythmic groupings. Each is hearing the entire complex of sound, in the sense that his eardrums and nervous system are responding to it, but each is perceiving—that is, is consciously aware of—different aspects of the sound.

Read more at https://www.stereophile.com/content/subjective-loudspeaker-testing#0yOF1tMqrEqD5jEl.99

Huh.

BDP24's picture

...is stated by JGH in his text. He moved a set of loudspeakers outside in order to determine how the room acoustics affected the sound of the speaker, so that he could then "subtract" that room sound from the sound produced by any and all other loudspeakers in that room. Sound heard from a loudspeaker in his room = 3. The sound of the room itself = 1. The actual sound of that speaker (as would be heard outside or in an anechoic chamber) = 2 (3 - 1).

Bogolu Haranath's picture

Listening to loudspeakers outdoors can almost mimic an anechoic chamber testing, because of the distances of the reflective surfaces are usually far greater than most of the room boundaries ......... Hence, less influence of the room boundaries reflected sound ........ JA1 tests some speakers outdoors, if possible to move them outdoors :-) .......

When this article was written, JGH most likely did not have the modern day testing equipment and computer software programs :-) .......

Of course, the design and tonal balance of outdoor speakers is different from speakers designed for indoor use :-) .........

X