The 2011 Richard C. Heyser Memorial Lecture: "Where Did the Negative Frequencies Go?" Case Study 2: Loudspeakers

Case Study 2: Loudspeakers
During a visit to Canada's National Research Council many years ago, I noticed, stuck to the wall of the prototype IEC listening room, a page of results from one of Floyd Toole's seminal papers on the blind testing of loudspeakers. The scoring system was the one that Floyd developed and that I subsequently used for blind tests at Stereophile: "0" represents the worst sound that could possibly exist, "10" the perfection of live sound. On this scale, a telephone, for example, rates a "2." The speakers in Floyd's test pretty much covered the range of possible performance, yet their normalized scoring spread, from the worst to the best, was just 1.9 points. Other than some pathological designs, the audiophile speakers I test for Stereophile probably cover a range of 0.5 point on Floyd's scale; but, as I pointed out in my 1997 AES convention paper, our reviewers' ratings generally follow Floyd's findings: that people tended to prefer speakers with flat response and controlled dispersion.

But not always.

There are vocal advocates of high-sensitivity horn speakers. There are equally vocal advocates of low-sensitivity, large-panel speakers. The horn advocates, among whom was Stereophile's founder, the late J. Gordon Holt, enthuse about these speakers' "jump factor." I used to ask Gordon what he meant by that: was it dynamic range? No, it wasn't. Was it low distortion? No. His reply was always that there was something about the presentation of these speakers that more closely resembled what he experienced from live classical music.

What about panel speakers? Being dipoles, their interaction with the room is very different, so maybe that's why their advocates love them, not even considering the fact that, with a large panel, the listener is almost never far enough away from the speaker to be in the farfield, the result being more bass—which everyone likes, at least in two-channel audio. Of course, there are no cabinet resonances because there is no cabinet, and perhaps people's preference is due just to the fact that he very large radiating area inherently ensures more linear behavior when moving the same mass of air as a much smaller cone driver. But panel speakers' impulse responses look terrible. Well, I say "impulse response"; what I actually publish and use for my reviews is not the response of a loudspeaker to an impulse, but the impulse response calculated by cross-correlating an MLS signal or a chirp signal, based on the loudspeaker being a perfectly linear system. (John Vanderkooy has investigated the shortcomings of this assumption.)

Perhaps, then, the problem with measuring panel speakers is that the calculated impulse response does not accurately reflect the behavior of a large panel in the same manner as it does a conventional speaker's. At the 1989 AES Convention in New York, mathematician Manfred Schroeder discussed Chaotic systems in audio and mentioned, in passing, that a Chaotic system tended to produce subharmonics. And indeed, when I have measured panel speakers I have found that they produce subharmonics. You feed the panel speaker a 1kHz tone and in its output you find a 500Hz tone as well as the usual distortion harmonics at 2, 3, 4kHz, etc.

So in a large panel speaker, while the average position of the diaphragm follows the driving signal, the motions of the individual elements of that diaphragm are Chaotic. The speaker sounds much better than its measured behavior suggests, as all that randomness is integrated by human hearing's latency.


Live vs recorded; Steinway D vs Vivid GiyaG1 speakers

Or do people like the sound of panel speakers because the intensity of the sound they produce—the sound power per unit area of the radiating surface—is closer to what people hear live? This question was triggered by a live vs recorded demo in which I took part in 2009, in Maryland. Two pianists performed for an hour on a well-prepared Steinway D grand piano, which I recorded at 24/96. Then, after a break, the audience heard the recital again, this time played back through a system. (This meant miking the piano far more closely than I would do for a conventional recording, to avoid a double dose of the room acoustic when the recording was played back.)

The system was full-range, with low coloration and sufficient dynamic-range capability that I was able to match sound-pressure levels. But even though the SPL and the sound of the piano matched closely, listeners felt that the reproduced sound just didn't have the "bigness" of the real thing. I wasn't sure what they meant by "bigness." Yes, the piano's dispersion would excite the room very differently than would a pair of speakers, but perhaps these listeners were suggesting that they were aware that the intensity of the sound was not correct, even when the SPL and response were correct.

So when people say they like panel speakers that measure poorly, perhaps they're responding to those speakers' more accurate intensity, at least when it comes to large sources.

Food for thought?

Share | |
COMMENTS
ChrisS's picture

Who's doing the listening in your tests, JRusskie? Do you know any 18 year old musicians? Oh, of course not...But you probably keep company with a bunch of construction guys (lucky you!) with damaged hearing. They should all find that there's no difference between any products.

ChrisS's picture

Once again, comrade JRusskie, you are on your Quixotic journey down that twisty, winding path for "truth"... Being an upstanding citizen of the former-USSR, you should know about "truth", right?

DBT is SCIENCE, and SCIENCE is ALWAYS RIGHT, especially during the heyday of the Union of Soviet Socialist Republics and you've been hanging on to this idea since the dissolution of the USSR. Hanging on to something you really want to be TRUE.

Comrade JRusskie, just because you want it, doesn't make it so.

Ariel Bitran's picture

just wanted to let you know i haven't forgotten about you.

i've been south of the equator spending time with my father and brother, but now that I'm back in the Stereophile office, i'll answer your question in full a little later.

peace out homeslice.

be nice.

Regadude's picture

Asking Johnny to be nice... Wow, you are very optimistic Ariel! 

GeorgeHolland's picture

Ariel, how about doing your job and deleting post and start placing a 30 day BAN on Regadude and ChrisS for spamming this whole article day after day with nothing less than pure drivel and taunts? I've seen kids frums better moderated than this fiasco but it's already turned into a child's forum the way they act. You would think they had some dignity but we haven't seen any yet.

Ariel Bitran's picture

I believe the internet is pretty much open range, so I'm happy to just watch folks speak their minds using whatever manner to express themselves.

one thing that bothers me is when a member is unneccessarily rude to someone who is being respectful or even innocently posting unknowing of the snark they may receive. in those instances, I say something so new users are not discouraged to continue posting.

On the other hand, if you're being provocative with your initial comment, I can only expect others to react to your provocation. 

******

The other policy I encourage and have stated many times is as follows 

"don't feed the trolls"

******

Finally, we do delete comments from ANY member we feel is totally off-topic and insulting to an individual member, as we want the conversation to continue and bring light to the subject  at hand rather than turn into a name-calling game. That being said, I'd like to delete Regadude's comment above since it just makes fun of Johnny. Also, I probably shouldn't have typed be nice. i dont know what i was thinking. I think i just wanted a rhyme in there, and also to include a general suggestion of positivity, but deleting RegaDude's comment would delete your reply, and this is a conversation I'd like to keep around.

Regadude's picture

What did I do? In response to your comment of "be nice" to Johnny, I merely joked that you were asking a lot. A lot, because Johnny is often not nice. He is that way, that is a fact. I have no responsibility for his behavior. I just pointed it out.

Don't shoot the messenger!

ChrisS's picture

What is reliability?

Think, Georgie, think.

Ariel Bitran's picture

the two links provided earlier giving examples of some DBTs.

I found the matrixhifi test to be ignorable: who is the sample? how did they select these people? how are they representative of a population of listeners as a whole? in order to gather significance from these these tests, the first and most important step is determining your sample, sample size, and how you select your sample. this just seems like a bunch of friends having fun. also, since there were multiple components being switched at the same time, system synergies could have been the cause of the weaker sounding more 'hi-fi' system. maybe those components weren't right for each other, but the cheaper system just sounded better. at least in ABX, they only changed one piece at a time

what i found interesting in the ABX test was the user's ability to control the change of system component themselves. this helps eliminate the idea that the listener might feel like they are getting 'duped' or constantly searching/guessing for the difference.

Also regarding the ABX method, the # of times a difference was heard was 33. the # of times no difference was heard: 29. Interestingly, cables were the least discernable. 

DBT is time consuming and for signnificant results you need a large sample size (to represent a large population of listeners). With a small sample size, as in both of these tests, you risk a greatly flawed hypothesis and will lack confidence in your results. 

I don't want to whip out my textbooks, b/c i have other stuff to do, but 17 listeners is not nearly large enough of a sample size to even represent a population of 70,000 Stereophile readers (for example). Then we run into an even bigger problem of "who" is selected-->ie what type of sample you are trying to represent.

I've heard repeatedly that H/K does have a successful DBT model. I'm sure it takes them years to perform each experiment, and it is wildly expensive and time consuming. You need a large sample size for any of this stuff to matter, not a few dudes in a basement.

GeorgeHolland's picture

More excuses from Stereophile?  Why am I not surprised. You scoff and blow off any attempts at doing a DBT by the people I linked to yet when it comes to "reviewing" cables and amps it's suddenly okay to accept a sighted biased review as being true?  Laughable sir but then I suppose you being an emploee have to toe the line. So be it.

I give up. Go on and trust your sighted "tests" while ignoring a DBT or even a SBT. I know your boss won't let you do any. He "knows" it all. *eyeroll*

Regadude's picture

Hey George! Ariel provided you with a more than satisfactory answer. So why are you still complaining? You never heard the expression "agree to disagree"?

Ariel is right; sample size is crucial. Anyone who has a basic knowledge of statistics understands this. 

JohnnyR's picture

George was talking about simply doing a DBT or SBT among friends like he linked to. Let's say you and 5 pals think the "Humungo" amp  blows away every other amp you have listened to. So you set up a simple DBT or even a SBT to see if you can RELIABLY pick which amp is which WITHOUT sighted bias. In other words using your EARS???? I have read on the forums here how much people rely upon "What I heard" yet don't trust a simple test to prove that they can.

So lets say the "Humungo" amp is just simply "liquid, lifts 400 veils and makes the back ground blacker", what ever silly terms you wish to use, If so then you should be able to pick it out 100% of the time doing a DBT. If not and lets say you only choose right 50% of the time then that's proof you were only guessing and couldn't tell which was which. So much for saying sample size is critical. If no one can pick the "Humungo" amp from the other one then where are your rationalisations for saying it's better?

I can tell though that none of you have even bothered to try a SBT or DBT so it's all a waste of time speaking to the peanut gallery. I love how much money you both have WASTED on pricey products that only look good and sound the same. Funny as hell, go on spending YOUR money I love ityes

ChrisS's picture

JRusskie just beat himself with his own thinking...

Regadude's picture

 

Johnny wote:

"I can tell though that none of you have even bothered to try a SBT or DBT so it's all a waste of time speaking to the peanut gallery. I love how much money you both have WASTED on pricey products that only look good and sound the same. Funny as hell, go on spending YOUR money I love ityes"

How do you know what I've bought with my money? I will go on spending my money (but not on your plywood speakers you make in your basement), that's what it's made for!

 

JohnnyR's picture

.a fool and his money are soon parted.

Regadude's picture

Well I am not that much of a fool with my money. I have not yet bought any of your JohnnyR brand speakers! 

ChrisS's picture

"...came out to play,

Georgie Porgie ran away!"

Delusions of absolute truths cloaked in "science" aren't much to hide behind, eh Georgie...

I hear our Man of La Mancha, JRusskie, is looking for his Sancho Panza to accompany him on his journey in search for TRUTH.

Happy trails!

ChrisS's picture

Georgie, Look at all those scientists backing you up!

If you take a college level research methodology course, Georgie, you may not appear so laughable. At this point, there appears to be very little knowledge in what you say about reviewing audio products.

GeorgeHolland's picture

Here is how you review audio products........ looks in Stereophile and buys whatever the product of the month is. Spends too much money but doesn't care. Goes online and talks like a 5 year old and spews insults and THINKS he's smart. Case closed.

Regadude's picture

George, name ONE SINGLE product that I have bought. Just one. You and Johnny have this little fantasy in which you think you know everything and everyone. 

How about you list your gear? If you are so good and knowledgeable about buying audio products, do share with us the products that are George Holland worthy. If you actually have any audio gear...

GeorgeHolland's picture

ZzzzzzzzzZZzzzzzzzzzzzzz sorry but conversing with the likes of you is boring and makes my brain hurt considering the amount of BS you spew,

ChrisS's picture

Take one course of college-level research methodology and call us in the morning...

rl1856's picture

Do you like what you are hearing ?  If no, move on until you do.  If yes, then shut up and relax.  This is a hobby focused on the enjoyment of the creative output of artists.  It is not about how many proverbial angels can dance on the head of a pin.

Go listen to MUSIC !

hnipen's picture

Thanks John for a very interesting and exciting presentation, lots of interesting information here and I'm surprised, to say the least, from the lack of positive feedback.

There are many who are skeptical to some of the ways of doing measurements in Stereophile and in some ways I'm one of them too, especially the way speakers are measured so close, large array speakers like the bigger Dunlavy's and some others will not sum up very nicely in this way. We do, however, not live in a perfect world and Stereophile cannot afford an anechoic chamber, so this is probably the best they can do.

I wish John would share more of this kind of information as he has gathered lots of knowledge during a long interesting career at Stereophile and other places.

Go on John :-)

Merry Christmas

Cheers harald

absolutepitch's picture

John, thanks for getting this lecture pre-print available for us to read. I have been looking forward to this.

I agree that there is a lot fo information combined into one lecture that anyone would need a lot of time to learn and understand the details. Pardon me for paraphrasing some of your words below.

Regarding the null result of DBTs, your description of the interpretation is in agreement with what I remember from statistics classes. I might add that a statistician would include a probability value or confidence band with the interpretation (something to the effect that 'the null hypothesis of no-difference-detctable is accepted with high probability'), and equally for the case when a difference is detected with high probability. I personally think DBTs should be done for product reviews, but agree that valid DBT's are difficult and time consuming (expensive) to do correctly, as Dr. Toole has shown in his writings.

The example of the 'backwards' impulse being not agreeable to listeners is something I have noticed in reference to digital recording. It's a wave form that does not occur naturally in music production, so reproducing it should sound 'bothersome'.

I also agree with the previous post, that more articles like this would be welcomed, to further highlight how complicated this field really is.

bernardperu's picture

I have read your essay with great pleasure (all of it!) and I think it is a great example of the Liberal Arts and Science coming together. In the end, it feels like a piece of applied music philosophy, which I find fascinating. It also seems to be free of busines-oriented interests, as your opinion on cables clearly suggests. It is awesome and very unsual to meet an accomplished person who gives priority to his passions and principles over financial interests (as also expressed on your 2012 writing on the CES and Las Vegas). 

I consider myself to be an audiophile that turns off the lights and tries to connect his emotions with the music with a very relaxed mind (this seems to be a category in itself, as the un-relaxed passive listeners who cannot focus on the music on a mid to long term basis tend to be very opinionated). Having said this, I recently purchased a pair of Class D mono amps that can clearly connect me to the music (Hephaestus brand). I have not ever listened to amps which are over 15k. Within similar prices, class D seems to be the better choice (but how relative this can be, Jon!)

I will continue to follow your writings with deep admiration and I thank you for making a difference on my musical experience (which is passed on to my girlfriend and my child). 

 

Bernard

GeorgeHolland's picture

Clueless you are if you think Mr Atkinson is something special angle

ChrisS's picture

Yoda you are?

Andreasmaaan's picture

It's a pity that some proponents of DBT as the only valid methodology have used the comments thread here to launch personal attacks against JA. Personally, I found the lecture fascinating and thought-provoking, and I thought that the nuggets of personal history provided a powerful context for the thoughtful opinions expressed. For better or worse, Stereophile doesn't restrict itself to DBTs as their only reviewing tool, but JA does measure every piece of gear his reviewers review - a practice which ensures that the opinions of the reviewers are grounded in objective data, or otherwise as the case may be. I'm not sure why this approach, coupled with a reliance on an income stream from advertising, seems to place JA in line for so much personal vitriol. If similar attacks were levelled at me in my professional life, I'd be mortified and enraged.

To cut what is risking becoming a lengthy expression of indignation short: thank-you JA for a wise and thought-provoking read.

JohnnyR's picture

WRONG!

"For better or worse, Stereophile doesn't restrict itself to DBTs as their only reviewing tool, but JA does measure every piece of gear his reviewers review - a practice which ensures that the opinions of the reviewers are grounded in objective data"

Do you even READ the reviews? Seriously dude. Stereophile DOESN'T do DBTs at ALL! Atkinson has said repeatedly that they are "too difficult to do". Show me ONE single DBT he has done in a review please. Plus he does NOT "measure every piece of gear his reviewers review". Cables, power cords, record demagnitizers, just to name a few. You obviously have been reading another publication NOT Stereophile.

Pages

X
Enter your Stereophile.com username.
Enter the password that accompanies your username.
Loading