Hoisted on your own petard?

After completing a PhD in electrical engineering at Imperial College London, Floyd E. Toole joined Canada's National Research Council (NRC), where he would stay for more than 26 years doing audio-related research. He continued his research at Harman International after leaving the NRC in 1991. When Toole left Harman in 2007 (footnote 1), Harman kept the work up under NRC alum Sean Olive (footnote 2)—which fact surely has much to do with the excellence of their current loudspeaker lineup.

The importance of Toole's project is hard to overstate. His goal was to provide a scientifically rigorous foundation that could inform choices made by audio designers, especially designers of loudspeakers. He succeeded. Drawing on his own research and the research of others, he established a template for what I call the "classical" loudspeaker: flat frequency response; excellent, well-controlled off-axis behavior; nonresonant cabinet, etc.

Toole's main technique was to carry out blind listening tests over many years with many subjects and analyze their preferences statistically. He learned that when it comes to loudspeakers, people mostly like the same things. As they get more training as listeners, they still like the same things; they just become more certain about their preferences.

The fact that there tends to be agreement between trained and untrained listeners adds depth to the research—it's much more than a mere survey of broad consumer preference—yet the work does not attempt to capture individual variation. It averages over preference. That's what needed to be done—really the only way forward, especially at the time the work was done and maybe still. And yet, while a loudspeaker designed to the classical Toole template will sound the best to the largest number of people, some people will prefer something different. That's not a defect of the research—it is what it is—but it is important to note what it isn't.

Some of the most passionate, deeply committed designers in the audio world refuse to buy in to the classical model. Their creations may lack the broad appeal of a classical loudspeaker, but many of them are very good at certain things and appeal to the significant subset of audiophiles who value those virtues over others. As the late Art Dudley wrote in one of his last columns, "From its acoustical beginnings, when two incompatible forms of physical media—Edison's cylinders and Berliner's flat discs—slugged it out for primacy, domestic audio has attracted an almost incalculable number of iconoclasts, heretics, mavericks, nonconformists, lone wolves, enfants terrible, and hidebound kooks. Because the above are among my favorite people, I don't have much of a problem with that state of affairs." (footnote 3)

Nor do I—I like the fact that the world is rich and varied. There is no single path forward but, rather, many paths leading in many directions and ending at many vistas (footnote 4). You may not prefer the view there over another view, or the sonic perspective, but someone does, and if your mind and ears are open, you can enjoy it. Do we want to live in a world where everything sounds the same? I don't.

It's disheartening, then, when speakers (and other components) that so obviously do not aspire to classical behavior continue to be judged by classical standards. "That speaker doesn't have a flat frequency response!" shouts an all-caps critic on some online forum, about a speaker whose designer never aspired to a flat response. "This speaker has a resonant cabinet!" exclaims another, about a speaker with a cabinet that's tuned to vibrate in a particular way. "That designer is inept!" writes a third, about an engineer who has sold tens of thousands of speakers—perhaps more—and won awards.

I admire Toole's work, but I do not admire conformists who insist, often with insufficient self-examination, that everything be judged by the same narrow criteria. There may be a single best way to roast a chicken, but I'm glad different chefs use different recipes. We at Stereophile encounter this problem ourselves sometimes, especially in measuring but also in listening. A loudspeaker (for example) that is intended to have a certain sound should not, I feel, be panned because it sounds different from what the reviewer expects or prefers. The reviewer's job is to characterize, not to condemn. As Art often advised reviewers, "Tell us what it sounds like!" If a loudspeaker is a sonic outlier, we must tell our readers that, but we have no responsibility to condemn it unless its sin is grave indeed.

And when a component is measured, what should it be compared to? Here, again, the reviewer's job is inform readers when a component deviates from what is classically thought of as excellent behavior, especially when that deviation is likely to be audible (although it is often difficult to know). We must of course tell them when that flaw is especially egregious and without obvious justification.

But there's a difference between ineptitude and nonconformity. Confusing the latter with the former makes the world a more arid, less-rich place. One can pursue excellence without excluding passionate outliers.

It's especially disheartening when narrow-minded online critics use one aspect of our coverage—our measurements—to attack the other side: our subjective judgments. We're providing a complete picture; the two halves make a whole. You don't get that from our competition.

Broaden your mind. Seek perspective. Look at the big picture.—Jim Austin


Footnote 1: Today, Toole lives in L.A. and heads his own acoustics and psychoacoustics consulting firm.

Footnote 2: A report by Kalman Rubinson on taking part in listening tests at Harman's facility can be found here.—John Atkinson

Footnote 3: Art addressed this topic in Listening #207—his fourth-to-last column.

Footnote 4: Which assuredly does not mean that forward progress isn't possible.

COMMENTS
ok's picture

Well, I try my best
To be just like I am
But everybody wants you
To be just like them..

Mountain Goat's picture

If you want to buy coloring, non-flat speakers, knock yourself out. Just don't pretend like they're good.

Bogolu Haranath's picture

See, Stereophile article 'Down with Flat', written by J. Gordon Holt, in 1985 :-) ........

John Atkinson's picture
Bogolu Haranath wrote:
See, Stereophile article 'Down with Flat', written by J. Gordon Holt, in 1985 :-)

www.stereophile.com/asweseeit/138/index.html.

John Atkinson
Technical Editor, Stereophile

tonykaz's picture

How could we live without measurements? ( we couldn't ! )

Audio Magazines devolve into Bar Room claims & braggadocio without accurate measurements. ( like the guy claiming to "improve on perfection" )

Good Science is repeatably measurable.

Society advances as improved measuring impacts our Quality of Life.

Stereophile had two measurement scientists: Tyll & JA ( we need a back-up )

The subjective folks are confined to a world of neurosis, psychosis and disorganised anarchy of Life's great understandings.

John Atkinson is our keeper of Standards.

Tony in Venice

John Atkinson's picture
tonykaz wrote:
How could we live without measurements? (we couldn't!) . . . John Atkinson is our keeper of Standards.

Thank you Tony. Since I started reading HiFi News magazine in the late 1960s, I have felt that measurements level the playing field.

John Atkinson
Technical Editor, Stereophile

tonykaz's picture

In my corporate world work, measurements defined the playing field !

Every detail, properly measured, contributes to the resulting performances.

"Take care of the pennies and the Dollars will take dam good care of themselves"

HFNews & RR had a strong measurement foundation.

Stereophile continues to have that incredible foundation of accurate detailed data, kinda making it a Pro-Audio type of Journal but with Social based review staff ( except for Mr.JA1 & JA2, of course )

Tony in Venice

ps. Stereophile is unique. Across the board, there are only a small number of Analytically Managed Monthlies.

Long-time listener's picture

... and I am eternally grateful for the ones in Stereophile. They've helped me understand many things and helped me make some good buying decisions. But there's always room for improvement. Stereophile's measurements of the Dynaudio Special 40 did not show any off-axis peaks in the presence region (which I heard, and which bothered me a lot given my listening position), but the measurements on another website very clearly did. Stereophile also does not show distortion measurements for speakers.

supamark's picture

Stereophile doesn't have access to the anechoic chamber at Canada's National Research Council (an incredible resource). I actually find the measurement of difference between playback at 70 and 90 dB that Soundstage publishes to be the best measurement that Stereophile likely doesn't have the facilities to make - it's great for showing which speakers suffer from power compression.

For example, Magico speakers compress horribly in the treble across the board (A1 to S5, their Be/diamond tweeters can't even play moderately loud) while Vivid Giya speakers don't seem to suffer from power compression (or distortion above like 65Hz) at all - Mr. Dickie knows his stuff and imo is probably the most talented loudspeaker designer today. Vivid speakers are probably the least distorted non-pro market speakers available for less than the price of a house (I consider ATC a pro audio company first, then consumer). Magico really needs to go back to the drawing board on their tweeter (and probably how they load it) because for the price they charge that sort of performance is NOT acceptable.

JA's frequency measurements on the other hand seem to match very closely to the response from the anechoic chamber at Canada's National Research Council - excellent job JA! JA's impedence/phase measurements are also more clear, and the running dialog through the measurement section just adds to the usefulness of it. Genuniely hope JA is with us and doing the measurements for a long time.

Anton's picture

That was a fascinating read, appreciate the input on this!

Bogolu Haranath's picture

ASR also does extensive speaker measurements, including compression, distortion and non-normalized FR measurements etc. ...... Take a look at one of the recent measurements of Tannoy XT-6 :-) ......

Robin Landseadel's picture

Looks like a reaction to the Audio Science Forum. Good luck with that.

JoethePop's picture

More likely in response to Brent Butterworth's article in Soundstage!Solo.

Robin Landseadel's picture

For the direction. Seems similar to ASR's sound.

Jim Austin's picture
The final proof for this piece--the last opportunity to make a change--is dated 27 May. It takes several weeks to get from proof to published. Jim Austin, Editor Stereophile
Robin Landseadel's picture

Meaning that:

https://www.soundstagesolo.com/index.php/component/finder/search?q=The+Biggest+Lie+in+Audio&Itemid=107

Is not specifically relevant, though the kind of writing found throughout the Audio Science Review probably is. In any case, I can't be the only one noticing how frequently badly performing gear gets obsequious write-ups in Stereophile and Analog Planet. I guess bad value for money is regarded as a feature, not a bug, in these publications.

Bogolu Haranath's picture

The reviewers in Stereophile choose equipment based on their sound quality ...... They don't know the measurements in advance ...... So, 'hits and misses' do happen :-) ......

Robin Landseadel's picture

"The reviewers in Stereophile choose equipment based on their sound quality".

I suspect there must be other factors that help explain the much higher than average cost of that gear.

Bogolu Haranath's picture

May be because, 'if it costs more it should sound good ...... Why would anybody charge so much, if it doesn't sound good?' ..... Just kidding ....... Several Stereophile reviewers review reasonably priced audio gear .... Many relatively affordable components are listed in recommended components list ..... KEF LS-50 and Wharfedale Linton are two examples :-) ......

Jim Austin's picture

The only criterion for selecting equipment to review in Stereophile is interest from a reviewer or editor. End of criterion.

OK, there's this: A company must be reasonably well-established in the United States: If they sell through dealers, they must have several US dealers (five is the rule of thumb).

I try to spread out the reviews--to avoid reviewing too many products from the same company in a certain period of time.

In contrast to some of our competitors, Stereophile does not do "pay to play," in any form. No one pays us for editorial coverage, in any form. Our business side and editorial side are separate; they sell ads, we write reviews and other copy. We exchange emails. Occasionally we have lunch, or did before the plague hit.

So, why do we cover expensive products? Stereophile has been a high-end audio magazine from the beginning. The magazine has always covered the most expensive gear. It still does. Those reviews are popular. People read them. We also cover less expensive products.

Why do we give them favorable reviews? We give them favorable reviews when, and because, they sound good.

Jim Austin, Editor
Stereophile

Anton's picture

"So, why do we cover expensive products? Stereophile has been a high-end audio magazine from the beginning. The magazine has always covered the most expensive gear. It still does. Those reviews are popular. People read them. We also cover less expensive products."

We are Hi Fi enthusiasts, all. Why bullshit it up by appending "high end" to "expensive products?" That's what killed TAS and turned them into an utter crap-fest of pretention.

If gear aspires to perform as Hi Fi gear, when does this bullshit term "high end" begin to apply? Can we request the price points for where High End begins?

I'm not mad at you, I am mad at that craptastic term that brings so much condescension into a joyful pursuit.

We are so self-congratulatory, sometimes. I am surprised when I don't see some reviewers mention that a pea under their listening chair cushion threw off their entire listening session.

The same thing happens in my wine hobby. Bullshit "high end wine," really?

"I am in to high end audio" or "I am in to high end wine" are likely part of nature's way of letting us know someone is likely a douche.

End rant.

Jim Austin's picture

But I don't share your associations. To me "high end" isn't a loaded term. Nor is it, to me, very specific or even well-defined. The distinction for me is mainly one of price--although there's also the matter of what that buys: a large materials and construction budget. Regardless of how it sounds, that's a real distinction that most, not all, hyper-expensive products share.

By the way, your "Princess and the Pea" comment reminded me of this review I wrote a long time ago. I think it's a pretty entertaining read.

Jim Austin, Editor
Stereophile

Bogolu Haranath's picture

The SACD data produces the sine wave more closely than the CD data ....... Similar to close sine wave reproduction by the 24-bit data :-) ......

Bogolu Haranath's picture

'High performance audio' may be a more appropriate term than 'hi-end audio' :-) ......

rschryer's picture

"High-End" smacks of class distinction.

"Hi-fi" includes everyone!

Bogolu Haranath's picture

Hi-end sounds like somebody is 'mooning' :-) ......

Poor Audiophile's picture

"Those reviews are popular. People read them".
Jim, you give the reader(customer) what they want?! Oh my goodness!
I'm a man of modest means(hence my name) but I'm not upset over the expensive gear being reviewed. Sometimes I even read those reviews & learn. Imagine that! I've been an Audiophile since H.S. in the 70's & I still learn.I of course like the coverage of "budget products" & miss Stephen's "The Entry Level".Except for doing that type of column again, I say keep doing what your doing. Larry

Robin Landseadel's picture

"Corruption", now there's a loaded term. Am I insinuating that the Stereophile policy is "pay for play"? That would be the model for "Parade" magazine, the long running add-on magazine attached to so many Sunday Papers, where it's PR, recipes and jokes and the "magazine" reads like the the puff pieces were supplied by the PR departments and published unedited. Stereophile isn't doing that. One could also think of the lobbyists on Capitol Hill who write the laws that get passed by the politicians that were installed by those lobbyists. Stereophile doesn't do that either. There's Fanfare, a journal of classical music recordings where the policy is pretty up-front: buy an ad, we'll review your recording. Stereophile doesn't do that.

What is going on is that the actual relevance of Stereophile is limited in the first place, and a magazine has to do what it has to do to capture eyeballs, and as far as I can tell, the model of capturing eyeballs by virtue of selling glossy, underperforming unobtianium has reached its limit and now there is pushback to this absurd show. It's been quite some time since I've seen copies of Stereophile at any newsstands, the hobbyists appear more interested in guns and ammo these days than sound gear. Of course, it's been months since I've seen a newsstand, and I'm sure all periodicals have had seen their newsstand sales sail off the cliff, as the newsstands are shuttered.

I have been reading audio journals for 50 years, starting with High Fidelity and Stereo Review back in 1970. I had subscriptions to Stereophile and other audio journals in the 1990's, read a lot of different audio journals including "pro audio" journals back when I got paid to make recordings. Something changed, and it changed relatively recently, where gear was being produced at price points well beyond the means of the average reader of those audio publications. A lot of that gear was, from a technical standpoint, underperforming. Magazines like Stereophile were and are doing their best to promote that sort of gear. Audio Science Review was one of those responses to that situation. Amir Majidimehr:

https://www.audiosciencereview.com/forum/index.php?threads/a-bit-about-your-host.1906/

. . . the man responsible for ASR, has a deep background in the science of audio and noted many over-priced pieces of audio gear whose performance did not justify the price tag. Audio Science Review doesn't take ads. Audio Science Review is primarily concerned with performance. The folks at ASR did note you hoisting yourself on your own petard. They responded:

https://www.audiosciencereview.com/forum/index.php?threads/was-this-aimed-at-asr.14717/

You're going to be reading a lot more of this. It's only going to get bigger.

funambulistic's picture

Because no one really cares...

ASR is an interesting and informative site (I enjoy the "SINAD" competition amongst DACs) but (in my all-too-humble-opinion) is way too focused on "measurements first, listening second" (if at all). As for loudspeakers reviewed on that site, it is almost a guarantee that if the ONE speaker (never set up in pairs, mind you - yes, I know it is a Toole thing) briefly tested does not sound like a Revel/Harman product, with whom Amir is associated, it will not pass.

Jim Austin's picture

... when we attempt to figure out what's going on in someone else's brain, we end up mapping out our own brain for all to see. That seems to be the case with your post.

[This comment is a reply to Robin Landseadel's comment, above.]

Your analysis suggests a sense of panic, an obsession with the bottom line. But, while obviously I see the ad counts month by month--I have to to lay out the magazine--the bottom line is something I rarely think about.

There was a time when all the audio magazines were much larger than they are now, because hi-fi claimed a much larger share of mainstream cultural and domestic real estate. And yet, when I had a glance at Stereophile's numbers before accepting this job, I was impressed with what I saw. Few businesses, certainly in the print-publishing industry, have such healthy margins. John Atkinson left me with a very healthy magazine. My only challenge was to continue to offer readers a compelling read. No rear-guard, defensive action required. Indeed, I am under no pressure from anyone or anything (except deadlines). I admit to some surprise at this, but it's absolutely true: No one tells me what to do.

Whatever you see in Stereophile is what I put there.

The correct version of your narrative goes something like this: Over a few decades, the industry changed. Maybe it's because of income inequality, the concentration of wealth. Whatever the reason, the price of the highest end got higher. In some respects, this is regrettable. In others, not. I like that some people like music enough to spend the cost of a modest home on a pair of loudspeakers. I appreciate those values. Plus, while not all take full advantage, those companies that do have effectively unlimited budgets to play with, and they take full advantage by producing amazing stuff. Boulder and MBL, to take two examples, charge a lot for their products, but they produce amazing stuff.

What you and others object to, it appears, is that at no point has Stereophile taken a stand against high prices. While continuing to cover more affordable equipment, we've chosen to be mostly agnostic about that issue (although some individual reviewers do object to the bleeding end of the price spectrum, so they review less expensive stuff). For you, this appears to be a moral issue. I simply do not share your view, so the magazine does not reflect that perspective.

A few months back, we put a $600/pair loudspeaker on our cover. A few months after that, it was a $150,000/pair monoblock amplifier. The next time a product comes along that I feel offers the same value (and high-end quality) as the Magnepan LRS--and which has some visual interest, which is important for a cover--it too will be featured on the cover. Inside, while the balance varies from month to month, you'll always find products at a wide range of prices.

I suspect that the reason the spam filter caught your post was the advocacy for and link to Audio Science Review, which technically violates our policies. I'm leaving it in.

Yes, I'm aware of ASR although I do not visit it often. I'm glad it exists. The measurements it publishes are valuable; we too believe in publishing measurements. In apparent contrast to you, I don't see ASR as competition; I see it as another resource for audiophiles. (For a while, the proprietor was presenting misleading information about its business status, labeling it a nonprofit when it isn't one, but he fixed that. Now: "You should assume any and all things can happen to money you donate including lavish vacations, expensive cars, exquisite sushi, etc. You should donate because you value what you see on the site, not for any 'cause' that your money would be put toward.")

I am a scientist myself, a physicist with a PhD., a few peer-reviewed articles published in respectable journals, and an edited book. I was employed for 14 years by the American Association for the Advancement of Science. I'm married to a professor of chemistry who holds an honorary endowed chair. Science is big in my life.

My only regret about ASR--this appears to be more true of its most zealous community members than of the proprietor--is the narrowness of their perspective. Many refuse to acknowledge, first, that, as I have written before, when it comes to the communication of emotion through music, science does not have primacy, let alone hegemony. And second, that other people's values--the way they make decisions about, for example, what audio equipment to buy--is up to them, not you. This is a perspective that I have come to, from a background based in science.

Despite the email quote above, which insists otherwise, ASR seems to have a calling, a moral cause, at least in the minds of its most zealous participants. Stereophile's only cause is promoting the enjoyment of music in the home, at the highest level.

Jim Austin, Editor
Stereophile

John Atkinson's picture
Jim Austin wrote:

Over a few decades, the industry changed. Maybe it's because of income inequality, the concentration of wealth. Whatever the reason, the price of the highest end got higher. In some respects, this is regrettable. In others, not.


I wrote about the upward-spiraling prices of high-end components and the possible reasons for it in October 2018 -

https://www.stereophile.com/content/conspicuous-consumption

- January 2017 -

www.stereophile.com/content/price-event-horizon

- and March 2011 -

www.stereophile.com/content/upward-price-spiral.

John Atkinson
Technical Editor, Stereophile

Robin Landseadel's picture

I'm really not sure how to respond to Jim Austin's unusually long and confusing response to “You’re Probably Wondering Why I’m Here”. I would like to note that when I mentioned/linked ASR I had two reasons that had little to do with promoting the site and a lot to do with the topic discussed. The first citation was to point out to Mr. Austin that all these people were talking about "Petard" at ASR, in thread entitled: ”Was this aimed at ASR?” . I was providing a link to that discussion, now up to over 280 posts. The other was to point out Amirm's history and qualifications for the job he does at Audio Science Review. Having had my posts delayed didn't strike me as strange, I've had delays before on account of excessive editing of posts after initially posting. So I was surprised to find out it was held up because of the link to ASR. Very strange, and very, very defensive.

Another thing, I have been aware of expensive gear that made claims of SOTA from the start of my interest in audio. High Fidelity had a review of the original Infinity Servo-Statik, back around 1970. I realized back then that the Servo-Statik was a groundbreaking product, one that would influence speaker design for years to come. And I remember the shock ‘n’ awe surrounding the Vendetta Research SCP-2 phono preamp in the mid 1980’s. Both justified their price. My complaint is not as much for the extreme high cost of 21st Century High-End gear as for the high-cost gear that technically is not up to snuff. Which goes to the heart of your argument.

Jim Austin's picture

So I was surprised to find out it was held up because of the link to ASR. Very strange, and very, very defensive.

I can't take responsibility for the operation of our overzealous automatic spam filter. As I indicated, I liberated your message as soon as I noticed it. I was only speculating about the likely reasons it was held back.

Both justified their price. My complaint is not as much for the extreme high cost of 21st Century High-End gear as for the high-cost gear that technically is not up to snuff. Which goes to the heart of your argument.

Indeed it does. It gets to the heart of our disagreement, and to the heart of what has motivated Stereophile since is founding more than 60 years ago. What you call "technical," as in "technically up to snuff"--which, based on previous posts, I take to mean adherence to certain standards of classical measured behavior--is only one side of a complex picture. To you and other ASR supporters, it is either the primary or the sole criterion by which a component should be judged. As I have made clear--and as Stereophile made clear for decades before I became editor--we regard measurements as a key part of the picture but not the whole picture. Nothing confusing about that.

Jim Austin, Editor
Stereophile

John Atkinson's picture
Robin Landseadel wrote:
I was surprised to find out it was held up because of the link to ASR. Very strange, and very, very defensive.

As Jim Austin has explained, your post was intercepted by our spam filter and placed in a folder for moderation. This isn't unusual or due to you including a specific link to ASR. As spam postings always include a URL and our site is hammered with spam, the filter is set to be aggressive with such postings.

I check the site for spam and posts requiring moderation every morning around 7:30am, so if your post was submitted for moderation after that time, I wouldn't have seen it. Apologies.

John Atkinson
Technical Editor, Stereophile

Robin Landseadel's picture

Thank you for that clarification. I want you to know that I have always appreciated the work you do at Stereophile and doubtless always will.

Bogolu Haranath's picture

Would the spam filter catch it, if I post a link to 'Eargo' website? :-) .......

JHL's picture

...was among the most coherent in the thread.

The problems with sighted measurement bias, aside from the obvious, lie in interpretation, device classification, and good old why-are-we-even-doing-audio. Meaning: either a pursuit for the ear is *heard* or it's endlessly debated to no useful other conclusion I've yet seen.

The measurement is technically useful and at the same has almost no reliable, interpretative, explanatory, or predictive bearing, among consumers, on sound. As such the era we're in - identified by a surplus of data and data-slingers and an almost complete lack of practical use of abstract information in the wild - should be a temporary one, as dismal as it can be in so many places. Eventually it'll be parsed deeper, first into a hierarchy of useful importance, and later into some next research work that evolves beyond the present limited conclusions of what too many people falsely claim is The Science.

There is no The Science. There are fragments of knowledge.

Jim's remark was right on target.

RH's picture

Jim Austin,

I find myself frustrated with the amount of pseudo-science that is excused under a totally subjective paradigm - "I don't care if the technical claims seem like snake oil! I tried it! I heard a difference! The claims must be true!"

So I do my fair share of defending the legitimacy of blind testing and measurements.

That said, caveats aside: I agree with your piece here. It is a well written plea for considering other points of view.

As it happens, I find that I may be one of the outliers in terms of the "Floyd Toole" school of loudspeaker science. I have no qualms at all with the science. And yet...I don't love every speaker designed by that science. For instance I like and admire the sound from Revel speakers, but have never loved it or felt pinned to my seat wanting to keep listening to my whole collection. Where any number of other speaker designs...some similar, some that deviate quite a bit....have pushed my buttons far more.

For instance, the Devore O series speakers sure don't measure like a Revel, but man o man do I want to keep listening to my music on those speakers vs any Revel I've auditioned.

Which brings up a problem: What do I do with this experience? I know from the science that the Revel speaker designer who uses blind testing will point out "But your tests have been done sighted. That means there are variables of sighted bias involved. We know that speaker preferences can change when the speakers are hidden from view. Statistically, you are far more likely under blind listening conditions to prefer the Revel speaker over the Devore speaker."

To which I'd reply: Yes, absolutely! All of that is true. The problem is that, like the average consumer, I don't have the expensive scientifically rigorous facilities to do such a blind comparison. So what should I do?

Well, I could just say "Well, they've done the science and since statistically speaking I would be likely to select the Revel in blind tests, I should just buy that speaker instead."

But that still seems problematic. After all, reading the science led me to expect I'd prefer the Revel speakers when I auditioned them. But I didn't. But I loved the Devore speakers. This could be explained by pure non-audio cognitive/sighted biases in play. OR...I could simply be in the group of people who would prefer the Devore speakers even in blind testing. As you point out, the science hasn't determined this for any individual, like me, either way.

Which means, at least for some of us, it still makes sense to determine which speaker to buy through personally auditioning, rather than relying on the measurements conforming to the Revel-school. (And of course, it leaves the option open, and completely reasonable, for others who want to buy based on the measurements they have determined as a goal).

So, I'm left pondering: Why would I pick the speaker that doesn't move me under actual-use conditions, over the one that does, on the grounds I MIGHT choose it under conditions in which I'd never listen? (Blinded).

So I can certainly see why some people avail themselves of the best science on loudspeaker preference and design to guide their choices. Completely reasonable! But in my own case, I feel I have to go with my subjective experience with the product under conditions that relate to how I'll actually be using it. If it's an amalgam of my own biases or quirks, I can accept that :-)

Finally, I'm very much with you (and Art) in enjoying the Wild West character of high end audio. While I think the Floyd Toole et al science is hugely important and revealing, I have seen many say on its basis that all manner of quirky high end speaker designs that don't conform to these "best practices" have no reason to exist at this point. All I can say is I'm very glad these people are not the arbiters of what products actually get produced. There are plenty of audio products that I have had a ball listening to at audio shows, or which I've owned and enjoyed, that would never have seen the light of day if everything had been constrained through the goals of conformity to one school of thought.

Cheers.

Joe Whip's picture

and that is that we each have a very distinctive and different processor between our ears. Hence, we all have different preferences in sound. While I tend to fall on the objective side of things it is audio after all and we do have to listen. What good is a great measuring dodat if I think it sounds poorly. If a piece measures poorly but I think it sounds great and brings me joy, so be it. As long as it is well built and will last I am good. One of the posters above loves Devore. I don’t happen to share that opinion but so what? They make him happy. That is the point. Choice is good. Imagine if the only ice cream flavor was vanilla? That would be boring. Same with audio. Choice is good. If you want colored speakers, that is ok by me, as long as I have an option that more suits my taste. Life is too short to get caught up in these endless arguments and harangues.

michaelavorgna's picture

The notion that other people can dictate what we should buy to enjoy listening to music through is as absurd as trying to tell other people what shoes they must buy.

RH's picture

Although there are certainly extremists in the "objectivist" world, in most cases I find the measurements crowd aren't trying to tell people what to buy so much as promote information and tackle false or dubious claims, so that people can make choices advisedly.

It's not "you shouldn't buy product X" so much as "you should know that product X is making these technically dubious claims for the product, and here's why they are dubious..."

So for instance, the measurement oriented objectivist will have no problem with anyone liking vinyl records; they only object to bogus technical claims vinyl-lovers often come up with to justify the "superiority" of vinyl over digital.

Similarly, with speakers, they understand everyone has his own preference, but they'll take on claims that some speaker is "state of the art" which is a more objective claim. "You may like the speaker and that's fine, but don't fall for the marketing that this is state of the art or that it required that amount of money to get that performance" etc.

(That's the general tenor I get from objectivist types - I'm kind of one of them myself - though, yes, I have encountered some who can not see beyond their own strict criteria for good and bad).

ChrisS's picture

...so much of your energy and concern towards the 99cent stores!

Do you really believe all those items are actually worth 99 cents?

RH's picture

When you attempt a reductio ad absurdum, you have to get the "reductio" part right, or you end up being the one producing an absurdity, instead of making a point. ;-)

Until you can get over your fascination with strawmen there's nothing of substance to respond to. From previous experience, I think this ends here.

Cheers.

ChrisS's picture

...for the marketing of this type of consumer product is disproportionate to the whole retail industry.

Your other concern for what type of information you think we should be getting from hifi reviewers is also inappropriate.

Point- no one does "blind testing" in reviewing audio equipment.

No one.

RH's picture

So your point is: "Hey, if you care about dubious claims for audio products, why aren't you spending your time vetting any number of other consumer items, like those that fill the dollar stores?"

Does it really need to be spelled out?

Ok, here: Like every other rational human being with limited time and resources, I apportion my time to those things that interest me.

I'm here like everyone else because I'm passionate about hi end audio. So I spend a lot of time on the subject, looking in to the gear and information as I create my audio system. I'm not passionate about 99 cent flashlights, or packs of ball point pens, or batteries, or birthday candles, or most of the myriad items that fill up dollar stores. That's also why I'm not devoting my time to learning all I can about any number of other consumer items that you may name...motorcycles, jewelry, video games, wankel rotary engines...

I would certainly advocate the usefulness of skepticism and critical thinking for consumers when it comes to any marketing claims.

But the idea that someone is unjustified in spending more time on his own hobby because he isn't also simultaneously chasing down misinformation in the "whole retail industry" or everyone else's hobbies, or any other red-herrings like dollar store prices, is a true absurdity.

This is what I meant when playing with reductio ad absurdums. If your "reductio" doesn't actually represent the reasoning of your interlocutor, you only end up presenting your own absurdity :-)

BTW, I have not demanded reviewers use blind testing (it could certainly be informative and revealing in cases where it is practical, but it would often be impractical for most reviewers).

But that aside, your post suggests a curious logic:

"Point- no one does "blind testing" in reviewing audio equipment.

No one."

What point are you making?

Are you moving from the premise "no one does "blind testing" in reviewing audio equipment."

To:

Therefore: No one OUGHT to do blind testing in reviewing audio equipment?

Can you spot the non-sequitur? (Think: David Hume) ;-)

Bogolu Haranath's picture

See, 'Inductive reasoning and Deductive reasoning' in Wikipedia :-) .....

RH's picture

I presume you are nodding toward Hume's Problem Of Induction.

I was referencing his other famous philosophical "Problem." ;-)

Bogolu Haranath's picture

Yes, of course :-) .....

Ortofan's picture

... include the results of blind listening comparisons.

Why doesn't Stereophile do the same?

ChrisS's picture

..."blind" listening years ago.

Ended up being a preference test.

Not very informative when you're told Reviewers A and B prefer Product 2, but Reviewer C couldn't decide between Products 1 and 4...

Stereophile does what it does very well.

RH's picture

Stereophile does what it does very well.

I agree ChrisS.

No publication can please everybody.

But for those who appreciate subjective descriptions of sound - and I'm one of them! - Stereophile has some excellent writers. For those who want things measured, Stereophile produces plenty of measurements.

It's a nice balance.

Ortofan's picture

... say that was?

At least one of the 2019 issues of Hi-Fi Choice describes their blind testing methodology as a preface to one of their group comparison equipment test reports.

ChrisS's picture

Perhaps...

michaelavorgna's picture

What you describe sounds even more dubious than telling people what to buy - are you saying that other people have the power to tell us what we should think about the stuff we buy?

Toward what end? That's like someone saying to you "I know you think your kids are cute, but they're not."

Come on, this is hifi. Not some competition.

RH's picture

"What you describe sounds even more dubious than telling people what to buy - are you saying that other people have the power to tell us what we should think about the stuff we buy?"

???? I'm not sure how you pulled that inference out of what I wrote.

No, it's simply about sharing knowledge, just like people do on any subject, in any hobby, and in most human endeavors.

If you were about to buy an expensive diamond ring from a jeweler for your fiance and someone knowledgeable pointed out that it's a fake and explains how you can tell it's fake...wouldn't that be useful information to have?

If you were buying a car on believing the manufacturer's claim you'll get 20 more miles to the gallon, wouldn't it be useful to have knowledge available that the claim is in all likelihood false and you wouldn't actually be getting what you think you're paying for?

If you were going to take homeopathy for a serious disease, isn't it helpful that knowledgeable skeptics can explain why homeopathy's claims are almost certainly bogus, and WHY they are bogus?

If a manufacturer claims amazing changes to the audio signal based on their special technology, doesn't it make sense that making the skeptical case for why it is pseudo-science and unlikely to work as claimed could be useful?

"Toward what end?"

Toward the end of sharing knowledge. Knowledge is power, right?
Isn't it wise to make our choices advisedly (as in the examples above?).

I did not describe jack-booted patrol crashing down doors insisting on what you ought to buy. I explicitly said that's not the case I'm supporting. Rather, it's just about making information available so audiophiles who wish to avail themselves of the skeptical case, or the technical info, can incorporate it in to their buying decision.

Or not.

Entirely up to the individual. Personally, even though I'm still no electronics whizz, I've benefited greatly from looking at various sides of debates in the hi-fi world. I wouldn't want be stuck in some single echo-chamber bubble where for instance only purely subjective evidence counts, or purely objective.

I'm not sure what you'd actually have against the approach I've *actually* described (vs a strawman) and have no idea what you mean by a "competition."

michaelavorgna's picture

...to buy things I like and enjoy when they're used for enjoyment.

To each their own.

RH's picture

"Afraid?"

Why the baiting choice of words?

What is so "fearful" about finding out something you didn't know, or being exposed to another viewpoint (e.g. that X audio claim may not be true)?

Are you advocating a bury-our-head-in-the-sand when it comes to knowledge about audio gear? Surely you aren't...so why do you reply as if I'm not making sense while avoiding my points? (If you agree that it is good to have knowledge available in the various examples I gave...why would you make an exception for audio???).

And for some reason you are implying a false dichotomy: that those who may care to understand audio gear technology, or who don't automatically buy whatever marketing is attached to an audio product, must be "afraid" and unable to enjoy their equipment.
(Why else pose your response as if it were some alternative to what I've been writing?)

I've been in to hi-fi for decades and I've enjoyed taking in and sharing all manner of reports and views and information on audio, from purely subjective talk (I love it!) to more skeptical inquiries of audio gear. And yet, I "buy things I like and enjoy" just like anyone else.

I'm sorry, but there are whiffs of strawmen, false dichotomies and a form of subjective audiophile virtue-signalling ("See how those objectivists/measurement nuts can't just relax and enjoy the hobby like me?") in your replies. I hoped for a higher level of discourse.

Anyway, thanks for your response.

Jim Austin's picture

RH, I happened to notice that your post got caught up in our spam filter. I'm not sure why. Nothing about it makes it seem like spam to me.

Now that I've gone to the trouble of rescuing it, I thought I'd add a comment.

I think what some (including me) are struck by is your assumption that the opinions of others are less well-informed, or less well-considered, than your own. That they need rescuing from their own ignorance and that you are the person for the job.

Much of my writing in AWSI, including this piece, is precisely aimed at establishing that people who think differently from you are not necessarily ignorant. Some know more, some know less. But in this context--and in general--I think it's a mistake to assume inferior knowledge.

People who enjoy this hobby don't need rescuing. Most of them anyway.

Jim Austin, Editor
Stereophile

Jim Austin's picture

RH wrote to me. He's apparently blocked form the site and can't respond. I'm not sure why because all seems well from my perspective.

Until that's sorted--I wanted to acknowledge that RH's first post in response to my AWSI piece is generous and well-considered. Nothing to object to. Then, as often happens, under pressure of debate, things got a bit off-course. My response above was perhaps justified in light of his most recent post, above, but in a larger sense I missed the mark. RH and I agree more than we differ. Nothing to object to.

Jim Austin, Editor
Stereophile

RH's picture

Here's the reply to you that didn't originally show up:

"I think what some (including me) are struck by is your assumption that the opinions of others are less well-informed, or less well-considered, than your own. That they need rescuing from their own ignorance and that you are the person for the job."

Ah, thanks Jim. It looks like I have to clear that up.

I had figured that what I wrote would be automatically understood as conditional. That is: all the examples I gave ASSUME the condition of learning something we didn't know, to make the point. Not that "anyone claiming to be an objectivist or expert - let alone me! - knows more than you and is by default correct." But rather "IF there is useful knowledge to be gained, even if it subverts your current belief or challenges the claim made for an audio product...why not be open to it?"

In fact I made the claim even weaker: any individual audiophile doesn't have to be open to gaining more information or "hearing various sides of the debate." Even that is up to the individual, but it is at least worthwhile for the debates to happen, to get info from various sides including the skeptical case, so it's available for those who care to avail themselves of the data.

Further: I certainly did not position myself as any expert source of truth or audio knowledge. I hoped that was clear from my initial post about Devore vs Revel speakers, and further clarified by pointing out I'm no electronics whizz and that I have benefited from watching the debates - that is from the knowledgeable contributions of other people.

I'd bet most people who even comment here know more than I do, technically, about hi-end audio gear.

"Much of my writing in AWSI, including this piece, is precisely aimed at establishing that people who think differently from you are not necessarily ignorant."

Absolutely. I could not agree more! In fact it's been a sort of hobby-horse of mine to argue for just that position! I'm concerned about how prevalent it is for people to not think beyond themselves, beyond their own goals, desires, criteria. If we only view other people's actions through the filter of our own desires, values and goals, we'll see everyone else as irrational!

So in audio, for example and to use extremes to make a point: If a "subjectivist" says "the ONLY way to evaluate audio gear is by LISTENING" one could point out: "Well, actually, there's another viewpoint on how to evaluate audio gear. Here are the goals/desires motivating that viewpoint and here is the method used toward that goal - e.g. comparing measurements, controlling for variables in listening tests etc."

On the other hand when an "objectivist" may say something like "It's ridiculous to buy vinyl given the much higher fidelity of digital these days" or "A loudspeaker with that type of response has no reason for being, given how it deviates from best practices..." then I think it's worth pointing out the value assumptions being made by the objectivist and how it seems to be blinding him to the perfectly reasonable actions of other audiophiles who have perhaps different likes/dislikes/goals/criteria. (For instance, "accuracy" is not some Absolute Value written in to the universe; some listeners do not have accuracy as their strict goal in choosing what they listen to. And their gear choices are just as logical, given their values, as the objectivists choosing accuracy as his goal).

I'd already alluded to this by agreeing with you in lauding the variety of approaches in high end audio, and how I would not like to see all products filtered through some narrow objectivists standard.

Does this make my position more clear or reasonable to you?

Cheers!

michaelavorgna's picture

...is a failure to communicate.

I'm not implying or suggesting any of the things you came up with. One false dichotomy is the entire subjective / objective "debate" which is filled with more straw men than a Wizard of Oz convention.

My point is people do not need to be rescued from their preferences when it comes to hifi.

Edit - I fleshed out my point of view back in 2010 in an AWSI titled, "Why Music Matters Most", which you can find in these pages.

Jim Austin's picture

Why Music Matters Most

... and while we're on that topic, here's a relevant piece I wrote last year:

How Does the Music Make You Feel?

Jim Austin, Editor
Stereophile

RH's picture

"What we have here......is a failure to communicate. "

Fair enough. I'm trying my best to be clear. Part of that is seeking conceptual clarity, making sure confusing nuances don't get entangled.

To that end:

"My point is people do not need to be rescued from their preferences when it comes to hifi."

I couldn't agree more! But that just puts us right back to the point I made: Who is actually acting in the way you suggest?

I'm not sure I've met anyone, on any side of the debate relevant to Jim's article who are trying to argue against personal taste. It's virtually self-evident that one likes what one likes. (Caveats aside for now). That's why I pointed out those who advocate the use of measurements, controlled listening tests etc are addressing objective audio claims- e.g. "Does X in fact lower distortion?" or "Is this tweak likely to actually produce any change in the sound?" etc. The "knowledge" aspect, not merely the subjective taste aspect. (They may be wrong...there's a debate there...but if an audiophile says "I much prefer listening to music with the Shunyata AC cable in my system over the stock power cable" the response isn't to deny the subjective experience, but to address the proposition that it is an actual change in the signal that is producing this impression. THAT's where the debate tends to happen.

I re-read your article "Why Music Matters Most" and found somethings to agree with, in fact your general point. But I found the piece did so at the expense of sometimes simplifying to a fault or eliding over various issues. So for instance in the article you say:

"The ultimate goal of the hi-fi enthusiast is the enjoyment of pre-recorded music in the home."

Fair enough!

But of course our goals and desires for any hobby don't just exist in a vacuum. We are always balancing other competing goals, for instance "I only have so much money to devote to the hobby, I want to get what I think I'm paying for and prefer not to be ripped off."

Now, you DO acknowledge the issue "But wait—haven't I just thrown out all objective criteria? Doesn't that leave us adrift in a sea of sonic anarchy, with people simply buying whatever they like to listen to, with no way of determining who has the good, the better, or the best hi-fi? How can we justify how much to spend? How can we compete with each other? How can we decide?"

But then you just leave that and return right back to the original thesis, leaving the problem essentially unanswered...or at least the rational concerns of other audiophiles over the above issues undressed. It seems your solution is just to go back to the criteria of "Am I involved in the music?" for buying a piece of gear. Which doesn't address the competing concerns we've just mentioned.

If I'm buying a diamond ring for my fiancee, I could say "The point of the ring is in how it makes her happy and how it symbolizes my commitment to her." But that's not the only thing I care about. I actually care whether it's a real diamond that I'm taking months of my paychecks to pay for! If it's actually a fake and I could have had the same thing for $40 bucks, that's useful information in balancing my decision. I'll prefer to buy the real diamond, for the 10 grand, thanks!

Similarly, IF (<-- this is the debatable part!), for instance an expensive audiophile AC cable does not alter the signal in any truly audible way over the cheap stock power cable on my amp, BUT I have bought in to the idea that "I need to spend more money on AC cables for my system to reach it's potential" then I'm operating under a falsehood similar to buying the fake diamond. The diamond may produce a similar experience to owning a real diamond when I am ignorant about the facts, and the AC cable may produce in my perception "better sound" which I ignorantly attribute to the cable vs my own perceptual bias. But in both cases it seems that knowledge is power; we prefer not to be wrong about the facts and very, very few audiophiles say "I don't really care if I'm fooling myself" in spending many thousands on tweaks; they really want it to be true and believe they "work" in essentially the way advertised.

Now, not every audiophile may be motivated to more deeply investigating an audio claim. Many may stick with "If I perceive a difference and like what I perceive, good enough, I buy it." (In fact, I have some items I own purchased on those grounds). Totally fine of course!

But other audiophiles may be interested in what is going on, gaining more knowledge...about say how electronics work, the likelihood of a claim for a product like an AC cable, the problem of sighted bias etc. Personally I have saved myself money, and sometimes some audiophile angst (when you go down the route "anything can make a difference" that can lead to audiophile nervosa), by considering the skeptical case against various audiophile gear and tweaks, and by occasionally performing my own blind tests (e.g. with AC cables).

Now, please remember all that was conditional: "IF" it's the case that some piece of gear has a false claim attached to it, THEN knowing this can be very helpful to those with goals that can be aided by that information, juggling how to apportion their audio funds.

No one has to incorporate the debates or any such data or information into his own purchases. That's for the individual to decide. But there are enough of us in the hobby who find it helpful to hear the "objective" or skeptical case on a hi-fi issue that it's worthwhile to have the debates occur. And that seems to me to be the general motivation for those presenting a skeptical case, and it shouldn't be confused with "telling people what they must buy or enjoy!" That type of characterization would be throwing the baby out with the bathwater.

So...ultimately our agreement or lack thereof would revolve around what you actually mean to reference by someone trying to "rescue someone else from their preference." In one sense that's almost self evidently a silly goal. In another, it's an equivocation of important issues.

Cheers!

Jim Austin's picture

>>I'm not sure I've met anyone, on any side of the debate relevant to Jim's article who are trying to argue against personal taste.

I understand why you might think so because it's so preposterous. But I see it often. Saw it recently, somewhere: In answer to, "Different people like different things," a person posted statistics from a Floyd Toole work purporting to show that everyone likes the same thing! (Wait, was that Brent Butterworth at Soundstage? I think it was.) That was unusually explicit, but it's exceedingly common to see people arguing that, while different people might like different things, some tastes and preferences are superior to others.

But there's something else going on that's a bit more subtle, although not terribly. People who think they have science on their side very often simply assume that they have the upper hand. But, as I wrote a few months back, when it comes to forming an emotional connection with music, science doesn't have hegemony. Other approaches are equally valid--even if they don't come with a certificate of certainty. (And then there's the question of whether science actually is on their side.)

I love the Art Dudley quote in this piece. It's a fact that some of the most obsessed, passionate people in our world are outliers in the sense that they don't buy into Toole's statistics-based hi-fi. I like classical speakers, but I respect that.

Jim Austin, Editor
Stereophile

RH's picture

Hi Jim,

I certainly wouldn't deny that from folks who have a "best practices" model in mind (e.g. speaker design), there isn't some level of eye-rolling and scoffing at people who choose speakers that obviously deviate from their model.

But I have found that when engaged on the matter, these folks typically will say "Look, your taste is your taste, but don't tell me your speakers are accurate/SOTA." It really is what they see as the objectively false claims, "snake oil" and misleading stuff in high end audio that they tend to see as a target, not personal taste per se. At least, that tracks with my long experience engaging both sides and everything in between in this hobby.

When I mentioned arguing against personal taste I did say at one point "caveats aside for now." The type of caveats I was thinking of were in fact the work of Floyd Toole et al.

When I said you can't argue taste, the caveat is presuming we are talking about an accurate idea of our taste.

It all gets so darned sticky.

So...given the work of Toole and Harman Kardon etc:

If I am sitting in front of two different speakers and I express my preference there is a sense in which, given Toole's research, I could be wrong, and a sense in which I would not be wrong.

If I say "I prefer the sound of speaker A over B" a researcher would plausibly say "Actually, given our research, in a blind test where you are ONLY hearing the sound of the speaker, it is highly likely you'll actually prefer the sound of speaker B. In this sighted test, there are likely biases involved causing you to percieve A as sounding better,e.g what the speakers look like, expectations about a brand, or whatever."

So the researcher has grounds to cast doubt on my own report of "which speakers sound I prefer."

But insofar as my report refers to my direct experience, in THESE sighted conditions, and I say "I'm enjoying listening to speaker A over B" then I can't really be wrong about that (unless I'm lying to them). Whatever additional variables may be involved in the sighted evaluation, it's causing me to indeed, perceive the sound differently and enjoy speaker A over B.

If someone tries to argue that I'm NOT preferring speaker A, they have no grounds to do so. (Or, at least, can not appeal to the blind test research to do so).

Would you agree?

That picky stuff aside: most people in discussing these things will say, yeah if you in fact prefer A over B I'm not going to argue against taste. But...once you start making testable claims...!

Finally, you wrote in another recent response:

... when we attempt to figure out what's going on in someone else's brain, we end up mapping out our own brain for all to see.

That at least partially describes a phenomenon that I have found to be very important and bedeviling of so much disagreement, on the web and elsewhere, whether it's debates about audio, politics, religion, culture, you name it.

That is: we tend to be quite bad at diagnosing the motivations of people with whom we disagree.

We attribute our position to reason; the other guy's position to psychology.

After all...we've done the reasoning, right? We've followed the logical steps, we have reached the reasonable conclusion. But the other guy didn't reach this conclusion, he has a competing conclusion. Well, that can only mean he didn't get there by reason - I know the right answer when using reason! Therefore I have to think up some psychological motivation for why he just won't follow reason! "You're only saying this/doing this because you want to X, Y and Z...!"

Any time we feel the pull that we all feel to psychoanalyze the opposing side we should take stock of what it is like when they do that to us. Isn't it telling that when an opponent psychoanalyzes your motivations he's always wrong? It's always some lazy pop psychology, usually attributing to you the must uncharitable motivations and simplistic ideas? How fortunate that MY psychoanalyzing of the other guy is always right, and they always get me wrong! Or...could it be, I may be just as lazy and inaccurate as the other guy? That I'm actually showing more about my own motivated thinking than I am about the opponents?

Always best to stick to the arguments.

But, boy, the pull of psychoanalyzing is strong and hard to resist!

Cheers!

ChrisS's picture

You

Talk

and

Think

too

much.

You are not listening.

RH's picture

Thank you for you insight and advice, ChrisS!

Rest assured, I will take it in the spirit in which it was given ;-)

ChrisS's picture

...too easy to get trapped inside one's head.

Listen to the music that gives you joy.

Better yet, go outside and listen to the sounds of nature.

RH's picture

Again, thank you very much!

"Listen to the music that gives you joy."

*smacks head* Now you tell me? I've been doing it wrong all these years, listening to music that makes me miserable. I've spent so long building a large library of LPs and digital music that I loathe, and now it seems I have to start all over again looking for music that brings me joy. I'm not even sure where to begin...(tossing out any Tom Waits will be a good start, I figure...)

"Better yet, go outside and listen to the sounds of nature."

As someone who takes daily nature walks and who designs sound for movies with an extensive nature library, and who does field recordings of natural sounds, it never occurred to me to actually bother listening to the sounds of nature.

Again, much obliged for your concern and your council.

Cheers!

ChrisS's picture

Think too much.

Jim Austin's picture

Thanks for the note.

In response, all I have to add is that, first, Sean Olive himself has replied to this essay on Facebook, saying, in effect, "I've got no problem with it." Meanwhile, acolytes, including at least one whose name you might recognize, are interpreting Toole's results as saying that, no, in fact, people don't like what they like. They all like the same things. There are many others making similar cases on various forums. It's not a strawman.

They are blindered. It continues to be true, and undeniable, that many of the most passionate audiophiles energetically pursue alternative paths. They care little for the classical sound.

Jim Austin, Editor
Stereopile

AJ's picture
Quote:

as saying that, no, in fact, people don't like what they like. They all like the same things. There are many others making similar cases on various forums. It's not a strawman.
They are blindered. It continues to be true, and undeniable, that many of the most passionate audiophiles energetically pursue alternative paths. They care little for the classical sound.

Jim, with all due respect, there continues to be a disconnect here.
"Tooles results" are BLIND LISTENING. Trust ears, Just listen.
There is zero correlation between this and what some "passionate audiophile" claims. Why is that so difficult a concept?
A completely uncontrolled believe/see/listen/random volume "experience" is utterly different from "Tooles results".
If you want a relative "Toole result", it would be this:
http://seanolive.blogspot.com/2009/04/dishonesty-of-sighted-audio-product.html
More recent: http://www.aes.org/e-lib/browse.cfm?elib=19405
And https://www.pnas.org/content/105/3/1050
The science aka "Tooles results" is very, very clear on this.
The "classically" cited "Toole result" is from trust ears, just listen controlled testing.
The "passionate audiophile" is doing no such thing whatsoever.
They are "experiencing" the DUT, yes, for sure. There is zero correlation between the two.
If we simply see/call them what they are, different...there can be no conflict. Nothing wrong with either...until false equivalence creeps in.

cheers,

AJ
Soundfield Audio

Jim Austin's picture

I love that phrase. It expresses so well the gulf between our perspectives. From experiencing ecstatic music to "experiencing the DUT." It takes something profound and makes it sound silly.

I understand your position, I assure you. You (and others) are denying the validity of other people's experiences because those experiences do not occur under conditions that allow them to be statistically validated. Most people, though, don't care about that.

You'll never understand this unless you allow your perspective to shift.

Many things can be profitably considered from more than one perspective. This is one of those things. Toole's quantitative-subjective perspective is undeniably a useful perspective; for some purposes, it's the single most useful. But it is not the only one.

I'll write again what I've written several times: People buy audio equipment to enjoy music at home. The point of that is the experience of the emotion music can provoke. People find that some components achieve that more effectively than others. When it comes to the experience of emotion through music, Toole's quantitative-subjective approach is not the only approach that has standing.

Insist all you want that none of these other approaches can be statistically validated. I'll even agree with you, but that doesn't mean that they are invalid.

Now I'll move away from the philosophy toward something more, well, obvious.

Many of those passionate audiophiles I'm referring to are listening to mono horn systems with flea-watt amplifiers. They've been to shows. They know what Revels sound like. And Magicos, and Wilsons. And then they fly to France to listen to horns in the basement of some medieval building with amplifier built on a hand truck.

These people have devoted their lives to the recreation of profound musical experiences via electronic means and have arrived at the conclusion that the mainstream stuff just doesn't get it done.

That is of course just one group of nonconformists, but it is an important one.

With all due respect, if you think those folks would suddenly prefer a Toole-spec'd "classical" loudspeaker if only they would audition it blind--well, that's just not plausible.

And by the way: I think they, too, should listen a little harder--a little more intently--to better hear the virtues we hear in these more conventional loudspeaker designs.

Jim Austin, Editor
Stereophile

AJ's picture
Quote:

You (and others) are denying the validity of other people's experiences because those experiences do not occur under conditions that allow them to be statistically validated.

No, completely false.
Science says and I cite, the false equivalence between peoples "experiences" and "Tooles results". THERE IS NO EQUIVALENCE.
The two things are totally different.
I said neither is "right" or "wrong". But "different".
If you can provide contrary science to my tip of iceberg citations, the floor is yours.

Quote:

Many of those passionate audiophiles I'm referring to are listening to mono horn systems with flea-watt amplifiers.

I would argue that an extremely rare, pathological case. Reductio ad absurdum

Quote:

With all due respect, if you think those folks would suddenly prefer a Toole-spec'd "classical" loudspeaker if only they would audition it blind--well, that's just not plausible.

I would argue there's only one way to find out ;-). As you know, Harmans test can be mono or >.
Btw, I have my doubts the average Stereophile listens in mono. Far from it. As such, it would be fascinating to find what their ears actually prefer. It would eventually make zero difference once "experience" happens again, but "interesting" none the less.
Having done many a blind test with audiophiles, I know that ;-).

cheers,

AJ
Soundfield

Jim Austin's picture

You say it's false that you're denying the validity of their experiences and then call them pathological. Sigh.

These conversations are more for others than for ourselves. In that respect, I think it has been useful.

Jim Austin, Editor
Stereophile

AJ's picture

https://www.merriam-webster.com/dictionary/pathological

Quote:

3: being such to a degree that is extreme, excessive, or markedly abnormal

Once again simple misunderstanding of word use.

I agree, in person, this conversation would be clear, probably done in 5 minutes ;-).
I'm not "denying" anyones experience. To the contrary, I advocate for just that...AND discerning between "experience" and "Tooles results".

cheers,

AJ
Soundfield

RH's picture

Hi AJ!

As such, it would be fascinating to find what their ears actually prefer. It would eventually make zero difference once "experience" happens again, but "interesting" none the less.

Yes. If I've interpreted you correctly, you seem to be allowing for both the power of the blind testing for weeding out variables, and also it's limitations in terms of real world predictions and applicability.

The blind test results are quite good at predicting listener preference under those blind conditions, and no doubt do illuminate real findings in the effects of different sonic characteristics on preference.

But once out of the lab, under the conditions in which people normally evaluate, purchase and listen to speakers at home (sighted), how well do they predict user preferences, satisfaction, and long term satisfaction?

I don't know of any research showing a strong link between the two.
Anecdotally, I have not found that the Toole/Olive et al research, and even my knowing it, has predicted which speakers I like to listen to most. (In other words, I experience enjoying a variety of other designs, some pretty neutral, others less neutral and not on the Toole-approved playbook), more than the Revels, even though I easily recognize the competence of the Revels. So do I choose the speaker that, statistically, I would be likely to pick should I do a blind test? Or the speakers I ACTUALLY find myself enjoying more under the sighted conditions in which I normally listen? (I choose the latter - I'm not able to untangle all the variables - how much of it is actually the sound, how much other considerations, but I DO know that I want to just sit and listen for hours through one speaker, while the other leaves me unmoved).

And along the same lines: whenever there are polls done in audiophile forums asking "what speaker is your favorite?" audiophiles chime in saying they have owned X, Y, Z speaker for many years, often decades. You find almost life-long devotees to every design under the sun, from Maggies, to Quads, to various Horns, to classic British monitors, it's all over the map. And the owners report great joy and satisfaction with their choices.

So I find myself of two minds about the research. On one hand it is exactly the type of research we should want done - that is anyone who wants to truly understand what is going on in correlating variations in speaker design to subjective results and preferences.
And it is hugely valuable to any speaker manufacturer who wants to create speakers based on the research.

But when asked...to what end? IF ultimately the end is in the satisfaction and listening pleasure for the owner, THEN it just doesn't map on so neatly once you get outside the lab with all of life's messy variables and influences. Then, if what you call the "experience" happens at home, the experience the owner has listening to his chosen speakers, then it seems clear from user reports that a very wide variety of speakers can provide that "experience" making people happy in many ways.

(Which is why, despite my admiration and gratitude for the scientific research, I'm also ok, even happy with, the bit of wild west character of high end audio, with various manufacturers following their own muse and not one single playbook for speaker design).

Thoughts?

Cheers!

RH's picture

^^^ I meant to bold AJ's comment starting out my comment, like this:

"As such, it would be fascinating to find what their ears actually prefer. It would eventually make zero difference once "experience" happens again, but "interesting" none the less."

(However, when I try to edit my comments to correct such errors, my posts go in to moderation, so I'm doing it in this post).

ChrisS's picture

anyone?

Anyone?

RH's picture

ChrisS,

You've repeatedly voiced your displeasure with the amount of thinking going on here, especially in my posts.

It's perfectly fine of course if you have your own personal threshold for thinking through the issues being discussed here.
That's cool. We all have our own level of interest.

But what makes you think you can adjudicate how much someone else ought to write or think about audio? If someone else gets pleasure from these discussions, what makes you so badly want to apply your own standards to keep saying "you are thinking too much!?"

Consider that other people have their own ways of enjoying hi-fi, which involves not only hours of listening pleasure and system building, but also hashing out concepts with other *interested* audiophiles. That's why so many audiophiles congregate in forums, and in comment sections to discuss articles.

Consider that the author of the article we are commenting on, Jim Austin, has welcomed these extended comments, and has even cited my reply as "generous and well-considered" and said he was happy to see my posts here.

I really enjoy seeing others who are thinking through the issues that arise in Jim's piece as well. (I notice you do not come down on Jim for the thought and time that he spent on raising the very issues we are discussing, or for his engagement in the comments elaborating on the issues raised, but you seem strangely fixated on my posts).

It seems a good time to take your own advice:

Instead of constantly refreshing the comments page eager to voice your displeasure with how someone else is spending their time, why don't you concentrate on healthier ways to spend your own time? I hear that listening to music that gives you joy, or going for a nature walk, are nice ways to get outside your own head ;-)

You may not be able to resist that feeling now arising to reply, especially with a dig. But...let it go.

Take a walk instead. Or just spend time enjoying music. It will be much better for you.

And if that feels like being preached at...may you reflect on that. ;-)

Peace. :-)

AJ's picture
Quote:

you seem to be allowing for both the power of the blind testing for weeding out variables, and also it's limitations in terms of real world predictions and applicability.

The blind test results are quite good at predicting listener preference under those blind conditions, and no doubt do illuminate real findings in the effects of different sonic characteristics on preference.

Correct. There is no equivalence between "Tooles results" as oft cited by audiophiles and "their results".
Unless they are (inadvertently) talking about the type of "results" I cited from uncontrolled "testing" above. Then they are in agreement!

Quote:

Then, if what you call the "experience" happens at home, the experience the owner has listening to his chosen speakers, then it seems clear from user reports that a very wide variety of speakers can provide that "experience" making people happy in many ways.

Yep, bingo. FAR more than "sound" (Tooles results) are involved.
Unfortunately so many cannot discern between the two.

cheers,

AJ
Soundfield.

p.s. I will use "atypical" instead of "pathological" to avoid misinterpretation next time ;-)

Jim Austin's picture

I now cannot see much difference between our perspectives at all. Did you read the essay?

Jim Austin, Editor
Stereophile

AJ's picture

I did and I think we are somewhat in agreement. I simply try to use words as meant per dictionary, such as "sound", listen", "ears", etc, etc.

Quote:

It's especially disheartening when narrow-minded online critics use one aspect of our coverage—our measurements—to attack the other side: our subjective judgments. We're providing a complete picture; the two halves make a whole. You don't get that from our competition.

The measurements strongly correlate to the "sound" (aka Tooles Results) and the "subjective judgements" strongly correlate to the experience...of which sound is a part of...but NOT the whole. As a mountain of evidence indicates.
I think its great Stereophile provides both.
To each, their own.

cheers,

AJ
Soundfield

Jim Austin's picture

>> the "subjective judgements" strongly correlate to the experience...of which sound is a part of...but NOT the whole.

It is precisely true that the Toole approach aims to explore just the one aspect of that--as you put it, the sound. In the context of the experience--which, with music, is what matters--people sometimes prefer non-"classical" sound, and that's legitimate ... and that it is legitimate to evaluate the experience, and not merely the sound.

Jim Austin, Editor
Stereophile

AJ's picture

We agree! ;-)

cheers,

AJ
Soundfield

Bogolu Haranath's picture

Referring to RH's comment and as an additional note ..... Maggies, Quad and Martin-Logan eletrostats don't have 'flat frequency response' ...... Many people like their sound quality :-) .......

JHL's picture

Many of those passionate audiophiles I'm referring to are listening to mono horn systems with flea-watt amplifiers. They've been to shows. They know what Revels sound like. And Magicos, and Wilsons. And then they fly to France to listen to horns in the basement of some medieval building with amplifier built on a hand truck.

These people have devoted their lives to the recreation of profound musical experiences via electronic means and have arrived at the conclusion that the mainstream stuff just doesn't get it done.

Precisely. Absolutely precisely. Any good diner has little interest in the chain steakhouse in the strip mall.

Further, any appeal to The Science of what is actually one tributary of knowledge - as occurred immediately after your remark - denies other tributaries or is ignorant of them.

Incidentally, I'd like to see a thorough discussion of the *arbitrary* 1M (or 50") amplitude data versus what speakers actually do in rooms, which is to dome over. I'm not asking for one; just would like to see it.

And that's just per *amplitude*.

The notion that high order multiways with good simple loudness uniformity are not only Perfectly Scientific but *exclude* all others from producing their frequently better audible connection to the recording is not only transparently risible, it's continuously disproved.

Jim Austin's picture

>>Further, any appeal to The Science of what is actually one tributary of knowledge - as occurred immediately after your remark - denies other tributaries or is ignorant of them.

I'm a huge believer in science--including the science behind audio. It can remain a very important tributary without presuming to be the only valid approach. Only if you embrace scientific standards and require statistical proof are scientific standards the only applicable ones. That is a valid approach to understanding one aspect of the experience. What I object to--and what my essay was aimed at--is the claim that science is the only valid approach.

>>Incidentally, I'd like to see a thorough discussion of the *arbitrary* 1M (or 50") amplitude data versus what speakers actually do in rooms, which is to dome over. I'm not asking for one; just would like to see it.

One needs a standard, and the point here is specifically to exclude the room. It's a nice metaphor for the broader challenge. It's necessary in order to figure out what the loudspeaker itself is doing, because all rooms are different. Of course, when you measure at 50" on-axis, what you learn about is what's happening on-axis (at 50", but if you measured at 60" you'd learn the same things). One needs many other measurements besides that one to characterize the behavior of a loudspeaker.

Of course, what matters is what it sounds like in a room--a particular room--and Toole (and that scientific project, broadly) have invested effort to understanding the behavior of sound from particular loudspeaker designs in rooms. The ability to predict the sound spectrum--as a function of frequency and time--at the listening position, in a particular room of known characteristics--from anechoic (or quasi-anechoic) measurements is, you might say, the ultimate goal of scientific research on loudspeakers. I admit that my knowledge is not up to date, but last I heard, this had not yet been accomplished. It does however seem achievable.

When it is, it will still remain to figure out how that behavior affects the total experience of music. This strikes me as a far more complex thing--much less likely to ever yield definitive results, although the kind of understanding the scientific project seeks--predicting the statical preferences of a group of listeners, trained or untrained--to me seems achievable.

Jim Austin, Editor
Stereophile

JHL's picture

...versus scientific finding versus universal scientific veracity is where we get tripped up. We're also severely tripped up by interpretation.

Science - as often as the word is turned into mere blind assertion about what we *think* the science is, says, or does - is a open-ended pursuit. (Check me if I'm wrong about any of this).

Scientific finding, on the other hand, is a touchstone at such and such a point in the research. That's Toolism; a touchstone and only that.

But absolute scientific veracity really doesn't exist. And that's where audio finds itself, at least with regard to speakers making lines on charts. That notion that we're dealing with the "science" of speakers in 2020 as an absolute truth is completely misled.

This particular science, what many find to be The Science, is a sliver of the whole. In real sound that whole is talking to us loudly.

To illustrate the fragmentary nature of the thing, there is *also* substantial science that well confirms fifty years of anecdotal audiophile preference where speakers are concerned. It finds that over large pieces of time and by way of audio professionals, all the stuff we value in real sound from boxes is confirmed. It's a prior work in that sense, and it is, in effect, the body of Stereophile subjectivity.

In other words, we like in speaker sound what others like in speaker sound when it's *not* Toolism but is another piece of knowledge.

The conclusion? There is the scientific pursuit, which is inherently fragmented and must be flexible to remain scientific. There are truths found in *various* of these pieces of knowledge, and lastly, there is no The Science - there is no ultimate so-called scientific veracity whatsoever in 2020.

Lastly, there's also no practical mental translation between amplitude-centric data and the sound of the high end speaker. It suffers a *severe* want of interpretation. There is, however, that fifty odd years of real high end speaker experience. And it's not amplitude-centric at the exclusion of everything else.

Speakers as competitive arguments built from an incomplete (and flawed) single "science" in forums is pointless and offensive. That's what amplitude bench-racing - and its *sighted bias* - have made them, however.

AJ's picture

Based on other posts in this column I'm now convinced my previous response to you...with AES etc links, has probably been spammed ;-)

Oooops, maybe not if more than I can see above!

Jim Austin's picture
I rescued it. Jim Austin, Editor Stereophile
AJ's picture

And rather quickly! Thanks...I'll go easy on the links next time ;-)
Unfortunately, your NYT link about orchestra blind test is paywalled ;-(
Title flies in face of all studies I've seen...but won't link

michaelavorgna's picture

..."people do not need to be rescued from their preferences when it comes to hifi" is that their experience matters more than the opinion of someone else.

I find the diamond analogy an ill fit for any number of reasons because the quality of a diamond directly effects its value (well, leaving out the fact that the diamond market is artificially inflated). So if you buy a fake diamond for a real diamond price and try to resell it, you are going to lose money.

This is not the case with hifi - poorly measuring gear can hold its value on par with gear that measures well. That's because people enjoy both.

Those are the main points that interest me and I hope I've been clear in my response. In a broad sense, we appear to agree more than we disagree, I've been listening through DeVore speakers for many years, but I have to admit feeling uneasy when someone tells me that they have concern for the less well informed since most of the people I've encountered in this hobby are very capable of deciding what hifi to buy.

Cheers.

Edit: I just read Jim's response and will add that I've also found 'people arguing against taste' to be common. I've had plenty, as in lots, of people try to convince me that some of the gear I've owned is not worth owning. These same people appear to relish the notion that someone else got taken while they never would because they're so...smart?

Also, if we step away from hifi and look at just about any other thing you can buy - cheese, butter, mattresses, jeans, wine, homes, boats, shoes, art, TVs, etc. we find the same array, the same type of broad spectrum of taste and preference as we find in hifi.

I think it's important to also recognize that some people, many?, buy hifi for reasons beyond measured performance or how it sounds. If we look at the rest of life, this should come as no surprise.

prof's picture

The fake diamond retains its value, too, just as long as there's another sucker you can sell it to.

This is what the economists refer to as "a market with imperfect information."

RH's picture

Ha, I was going to make the same point! ;-)

I believe the diamond analogy holds (though any number of others could suffice - e.g. being duped in to thinking you have to pay triple price for a special gasoline that will make your car last longer, when normal priced gas would do the same).

But, like the diamond: knowledge is power.

IF, like the fake diamond, the claims for your expensive AC cable are fake and you could have gotten the same results from a cheap stock cable, then the fact you can find another "sucker" to sell the fake diamond/cable to is just beside the point.

The point is in the value of having the knowledge about what is really going on - in knowing that you can get the SAME performance without having to pay thousands of dollars for it. As I said, not everyone will use the information or accept it, but especially IF TRUE a lot of people will appreciate having this knowledge!

Which, again, is why skepticism can be valuable.

prof's picture

IF, like the fake diamond, the claims for your expensive AC cable are fake and you could have gotten the same results from a cheap stock cable, then the fact you can find another "sucker" to sell the fake diamond/cable to is just beside the point.

On the other hand, if someone starts going around telling people how to distinguish fake diamonds from real, this can really impact the resale value of the former (pissing off a lot of current owners in the process).

michaelavorgna's picture

Let's think about synesthesia ("union of the senses"). I had an art teacher in 1980 who was a synesthete - she saw colors when listening to music (among other things). Seeing as this was 1980, most students thought she was kinda whacky.

From Wikipedia:

"Research into synesthesia proceeded briskly in several countries, but due to the difficulties in measuring subjective experiences and the rise of behaviorism, which made the study of any subjective experience taboo, synesthesia faded into scientific oblivion between 1930 and 1980."

The study of this phenomenon arguably dates back to ancient Greece.

The relevance in this context, aside from the behaviorism angle, being what we hear and how we experience can be hugely, dramatically, different. So it makes perfect sense that different people prefer different things and it makes no sense to me to set one approach as the hifi standard against which everything else is judged. After all, "accuracy" is not an end in and of itself when it comes to the enjoyment of music. They are not even necessarily directly connected, which I think is one of Jim's points.

But I understand the desire to judge and believe we own "the best" and that we are Right (which necessarily makes other people Wrong). I'm not saying you believe this but some people wield their personal preferences, using "accuracy" like a weapon, to demean and dismiss other people's preferred means of enjoying listening to music on the hifi.

I have to ask, toward what end?

DukeL's picture

One of my sons perceives sounds as colors. He literally had no idea that everyone else did not until he saw a documentary on the subject.

So I'm comfortable with the idea that individual perceptions can deviate significantly from the norm.

John Atkinson's picture
DukeL wrote:
One of my sons perceives sounds as colors. He literally had no idea that everyone else did not until he saw a documentary on the subject.

I have a touch of synesthesia, in that some sounds elicit a sense of taste. When I played violin, the sound of the instrument so close to the ears led to a sour-tasting sensation - even when I played in tune!

John Atkinson
Technical Editor, Stereophile

Bogolu Haranath's picture

Sauerkraut taste? ...... Just kidding :-) ......

DukeL's picture

John, never again will I question your taste in music!!

Duke

Bogolu Haranath's picture

Tell me what music tastes like single malt Scotch ....... I'll buy that music :-) ......

RH's picture

Fascinating, John.

I've wondered about the prevalence or role synesthesia might play in the audio hobby.

I often experience sound evoking colors in my mind, and I've wondered if this is a touch of synestheia, or more exactly, Chromesthesia (sound in to color). Or whether it's just they type off association most people have.

For instance, when I hear many acoustic guitars, including my own, it evokes a warm, golden, sparkly tone in my mind. Consequently, one of my main tests for any loudspeaker is playing acoustic guitar tracks, including a recording of my own guitar. If it does not evoke that same timbral image, the same colors, then it sounds "off" to me and the sound is un-involving - a black and white version, or a silver version rather than warm and golden.

This also made me curious about whether synesthesia was reliable and repeatable. Is it something sort of automatic and independant of bias effects that the brain is doing? In other words, would the same speaker in a sighted test that can produce a "golden tone" also reliably be identified in a blind test because the brain automatically maps that color to that sound? I don't know. But I can say that whenever I heard a speaker somewhere that got me excited because it evoked the "right" tonal colors, when I could get that speaker home to do live vs reproduced comparisons with my guitar and other instruments, they did indeed sound more like the real thing, tonally.

But audiophiles and reviewers have long used some appeal to colors when describing sound. For instance, classic Conrad Johnson tube amps were often described as having a slight "golden/amber glow" in the upper mids/lower treble. And that is indeed what I always heard from my CJ amps.

ChrisS's picture

Stereophile already does a very good job at this....

JoethePop's picture

RH, I don't understand why anyone would get so defensive about what you wrote. A nice balance between objective and subjective and how important it is to have both. Sadly, it seems like in almost any subject these days you have to pick an extreme. There can be no middle ground.

RH's picture

Thanks JoethePop!

Admittedly some of us are battle-scarred enough from on-line skirmishes that we can jump to wrong assumptions. Someone makes even some objectivist or subjectivist noises, and we presume they likely believe whatever the previous extremists have argued.

Bogolu Haranath's picture

We don't have 'cancel culture' here at Stereophile :-) .......

Joe Whip's picture

I keep my mouth shut unless someone asks for my honest opinion. Even then, I try to be respectful. You are right, it is only Hifi. Some tend to take this stuff way too seriously, hope you are doing well Michael.

michaelavorgna's picture

;-)

Thanks Joe. We are doing well in these trying times. I hope you & yours are well.

Joe Whip's picture

Yes thanks. More time to listen and write. Be well.

CG's picture

Not a competition?!?

Since when?

michaelavorgna's picture

...in the next Olympic Games.

Bogolu Haranath's picture

If they don't play Kanye West, I'm not going :-) ......

Bogolu Haranath's picture

In the movie 'Indiana Jones and the Last Crusade', in the final scene the Grail Knight said to Indiana Jones 'He chose poorly but, you have chosen wisely' :-) .......

DH's picture

There's well regarded equipement out there that when measured, is shown to be poorly performing - high in distortion, etc.
Does that mean I have to "protect" someone from buying it? Not at all.

But I think people should know that their money is going for something with distortion that they apparently apparently hear as euphonic. Maybe they'd want an alternative if that information is available to them.

Personally, I don't want to buy equipment like that. I'd rather compare components that are objectively more accurate, and then pick the one I like. If I'm spending my money, I want it to go to something well engineered, and not something that doesn't actually reproduce what's on the source. Of course I want it to "sound good", but for the right reasons - that it is giving me true hi fidelity to source reproduction.

Solarophile's picture

Wholeheartedly agree with this idea. A good example of this I think is what Archimago wrote about this week and the mix of objective measurements with subjective opinion (Pass ACA). I think that is the kind of review most helpful for audiophiles combining technology with understanding of preference. Subjectivity combined with knowledge is much more powerful.

It's good to know when equipment measures well but no need to fault anyone for liking a product regardless. Plus it is good to have the data to judge whether something is hyped up.

Volti's picture

Measurements are very important to me. I couldn't design my speakers without them.

The problem I have with the "Classical" parameters, is that they are subjectively derived, even if from a large group of listeners over time, and they do not accurately reflect or adequately measure what I like to hear from my speakers.

My measurements are accurate and useful and also subjectively derived. They help me get part-way towards my goal with a design. The rest is done with my ears.

When I'm happy with a design, the measured performance of that design, especially when compared to the "Classical' standard, is almost of no interest or use to me or my customers.

Greg Roberts

Ortofan's picture

... do you attend concerts of un-amplified vocalists and/or acoustic instruments?
Do you play any instruments?
Is what you like to hear from your speakers based upon such experience or is it influenced more by how the playback of certain favorite recordings sounds to you?

Jim Austin's picture

I found this at the New York Times--relevant, at least in a rough, approximate sort of way.

To Make Orchestras More Diverse, End Blind Auditions

Jim Austin, Editor
Stereophile

CG's picture

It probably IS relevant.

I asked the resident expert on blind auditions about this. She said that the system described in the article isn't exactly how it was in her experience back in her school days. There, you not only had individual blind auditions but group blind auditions. That is, you didn't just sit behind a screen and play the music invisibly to the judges, but you sat behind a screen and played with other musicians behind the screen. That way the judges could also tell how you musically (and otherwise) interacted with the group.

Of course, there's lots of potential problems with this on a human level. But, that's a discussion for another time and place.

With audio gear, you also have interaction problems. This isn't witchcraft - much of the basic electrical interactions are taught in the first physics courses an electrical engineering student takes as a freshman in college.

How do the traditional measurements take this into account? Seems rather incomplete to me. Not inaccurate - incomplete. Kind of like that old parable about blind men describing an elephant. Every man had an accurate but incomplete description of the elephant.

Gary Dews BorderPatrol Audio's picture

The British audio mag Hi-Fi Choice ran blind group tests through the late 1980’s and 90’s hosted by no less a light than Paul Miller.

A listening panel of four people (I was lucky enough to sit on few) would listen to a system which was behind an acoustically transparent curtain. Mr. Miller would then change amps or DAC’s etc. The panel were asked to comment on each piece. PM would also do lab tests on the equipment.

The panel were usually very complimentary about tube equipment and in particular models from the UK manufacturer Audio Innovations. The Audio Innovations products usually had some of the worst test measurements PM had seen with high levels of harmonic distortion, poor frequency response, high output impedance etc, etc.

Mr. Millers conclusions were usually of the type (paraphrasing) ‘this amplifier was warmly received by the listening panel despite having some of the worst specs I have measured’.

If memory serves there were also more than a few solid-state amps that conjured up the opposite. ‘Despite a solid set of lab measurements, this unit failed to generate any enthusiasm from the listening panel’.

eriks's picture

OK, I'm rewriting this in the interest of not being a cynical jerk.

It is the lack of context, and lack of using your own measurements to inform the reader that is the problem. I run into this problem in IT all the time. The lesson I always give students: "Don't produce data without expert advice and context or you will suffer for it."

Let me instead try to be constructive. You've invoked Toole here. So, you seem to know what common and good speaker design practices are.

The problem I've had, and I can point to specific instances, is when you measure, and there are glaring issues vs. classical design principles and the reviewer ignores them.

For instance, the Totem Skylight.

https://www.stereophile.com/content/totem-acoustic-skylight-loudspeaker

You complain that readers took you to task. I would suggest that you could have avoided that by going back from your measurements and ask if the speaker showed evidence of the expected problems:

Excessive overlap and interference of the tweeter and woofer.

That would have avoided any criticism. Something like:

"After looking at the measurements, we listened below and off axis and found the sound colorful but we liked it!"

or:

"Below the tweeter axis the sound is poor, so pay attention to placement and tilt"

or:

"We were surprised that despite the off-axis measurements, the Skylight still maintained a stable and pleasant off-axis image.

Volti's picture

What if all audio components could be reduced to a set of measurements? Wouldn't that mean that we would all have the same sounding speakers, amps, DAC's, etc... - obviously the ones with the best/only specs?

The measurements would need to be complete enough to capture every aspect of what makes a component sound a certain way, and I’m pretty sure right now we don’t have nearly enough measurement capability to do that.

But if we did, there would only be ONE set of scientific measurements to build to, and that would eliminate all ‘flavors’ of sound from our components and each one that met the measurement criteria would sound exactly the same. It would be very easy to prove that a component is inferior – it simply doesn’t measure up.

There would be no more forum arguments! lol

Greg Roberts

prof's picture

What if all audio components could be reduced to a set of measurements?

...

There would be no more forum arguments!

You make it sound like that would be a bad thing.

DukeL's picture

From the article: " I admire Toole’s work, but I do not admire conformists who insist... that everything be judged by the same narrow criteria. "

I agree with writer Jim Austin.

Neither Floyd Toole nor his colleague Sean Olive claim that their measurements and analysis tell the whole story, though many assume they do.

From one of Sean Olive’s landmark papers, "A Multiple Regression Model for Predicting Loudspeaker Preference Using Objective Measurements: Part II – Development of the Model":

"LIMITATIONS OF MODEL

"The conclusions of this study may only be safely generalized to the conditions in which the tests were performed. Some of the possible limitations are listed below.

"1. Up to this point, the model has been tested in one listening room.

"2. The model doesn’t include variables that account for nonlinear distortion (and to a lesser extent, perceived spatial attributes).

"3. The model is limited to the specific types of loudspeakers in our sample of 70."

By way of example, imo Greg Roberts' controlled-pattern designs interact with normal listening rooms more favorably than the wide-pattern speakers Harman finds to be preferred in single-speaker auditions in the middle of a 21-foot wide room.

(Incidentally the verb from which the noun "petard" is derived, "péter" [pronounced "pet-ay"], means "to fart".)

Duke LeJeune
AudioKinesis

funambulistic's picture

I'm really confused here: wasn't Petard that bald dude on the Enterprise?

avanti1960's picture

shooting themselves in the foot by relying on measurements of speakers from which to evaluate them.
book smart is a grossly incomplete skillset in this hobby. you need also to be street smart and get out in the wide audio world where it will not be difficult to find exceptionally measuring speakers that are not inspiring and ones that have questionable measurements that sound exceptional.
personally i prefer some strategically executed cabinet resonances for example.
a little experience goes a long way.
of course we love / need the subjective commemts for no review of a musical reproduction device would be complete without telling us about how the device elicited the intended emotions while conveying the evokative art form of music.
watching a video of real time frequency plots of a symphony sans audio simply will not suffice.
we do need you to tell us what the speaker sounds like, but subjective reviews do have their limitations. some fail to capture the essence of the speaker and most are simply too kind. we understand though,staying well clear of those petards is a wise move.

avanti1960's picture

measurement labs and the speaker measurements processes are tools to help engineer and develop the product but they are not the final word.
the final word is validation through listening, subjective evaluation and iterative "seasoning" to arrive at the desired voicing.
if speaker engineers need subjective listening to complete the design and validation process before it is released, how can a published review be complete without subjective comments?

ok's picture

speaker measurements matter more than their electronics counterparts is that passive speakers do not employ negative feedback; this allows for huge differences in all parameters and simultaneously forbids unfair steroid competition. You get what you see all right - but what you see.. still not enough.

AJ's picture

I must admit as a speaker manufacturer I never considered giving them steroids.
What if they flew into a rage at their new owners?

cheers

AJ
Soundfield

Bogolu Haranath's picture

The photo on the top of this forum shows, what happens when 'speakers on steroids' can do, if we make them unhappy :-) .......

Jeff&#039;s340a's picture

Jim, Let's not focus on people with rulers who make comments that bother you.
I propose you look into your own product and ask why clear opinions are not the norm. Any way you care measure it, my query it comes up as legit.

supamark's picture

1. Mr. Austin - please get yourself a photo avatar so we can more easily find you in the post stream (you're level headed and well educated, that's good and need more of it in some of these threads).

2. can someone start posting content on InnerFidelity and Audiostream? Essentially no content in months... Hell, maybe even temp hire Mr. Lavorgna back if he's up for it to get some content up (not my fave writer, even told him so, but he is a capable reviewer and y'all need to get some content up or those sites will fold and they're worth keeping around). I'm sure there's something going on internally, and that it's none of my bee's wax, but y'all need to do something if you want those two brands to continue existing... assuming the publisher actually wants that.

oh, and bonus round - Covid ain't no joke, mask up people! Do it for your vulnerable friends, family, and neighbors and to get back to normal as soon as possible (my wife and I are recovering from Covid, and it IS worse than flu which I also had this year).

supamark's picture

It looks like point two has been answered (and those two sites closed down). will the archives be made available?

SteveR1's picture

Then - maybe instead of questioning the usefulness of detailed specs - a sensible person might want to question the soundness of one's ear and work on improving that.
I recall some iconoclasts at Stereophile reviewing state of the art amps with 1960's speakers [that weren't really that good even in the '60's]

Trevor_Bartram's picture

Try as I might I cannot over get the fact that every time I see the name Floyd E. Toole, my mind is instantly transported to the movie 2001: A Space Odyssey!
Probably a combination of Dr. Frank Poole and Dr. Heywood R. Floyd. Childish, I know, but I can't help it.
It may also have something to do with my life now being movie centric rather than music centric, due my poor hearing, oh well.

reponkic's picture

ASR has killed your publication, face it. No amount of butt hurt name calling by Jim will change that. Feel free to praise overpriced trinkets for sad fantasists, but the world has moved on.

RH's picture

Congratulations on lowering the level of discourse with your raging superiority over other audiophiles. That attitude is s great advertisement for ASR!

(Fortunately, you will actually find more even-keeled, even if critical, commentary on ASR concerning Stereophile).

reponkic's picture

...if people cannot handle the truth.

RH's picture

I'm a happy member of the ASR forum, so let's try and dial down the roaring triumphalism to a manageable din for a moment.

You think ASR has "killed" stereophile, and apparently feel great glee over this claim.

Let's think this through a bit more...

The work Amir is doing at ASR is wonderful and welcome. That said, an ASR they are measuring Amplification, DACs and now Speakers. And particularly with speakers, this work has just begun.

Meanwhile, Stereophile has been by far, the biggest public resource for measurements of hi-fi gear - Amplification, DACs and Speakers.
Decades and decades of those measurements have been made available essentially "free" now in the archives. For a long time even when measurement-oriented audiophiles and engineers wanted to mock a product for ineptitude, they would draw from Stereophile's measurements.

When you claim "overpriced trinkets" surely you want to have objective data to support your claim. After all, it would be pretty unscientific to just go on looks or have a hunch. Where are you getting your information from that Stereophile reviews any "overpriced trinkets?" I'd bet it's from Stereophile's own measurements! (ASR has barely touched most of the products reviewed in Stereophile, so it can't just be ASR's data).

Further, at the moment the ASR measurement project depends on the goodwill of owners to ship a pair of speakers to Amir. As grateful as we may be to those willing to do so, as a model for covering the vast range of hi-fi speakers being produced, especially from major manufacturers, this hardly bodes well. Right now, as one might expect on the model, it's generally a mish-mash of whatever someone will send Amir - cheap speakers, some a bit more expensive, old speakers, some new. It certainly adds up to learning more, but I see no road towards even coming close to the level of product measurements Stereophile has and can produce. You aren't likely to see anyone shipping their big Wilsons or whatever to ASR, whereas Stereophile routinely reviews and measures speakers from small and cheap to massive and expensive and *attempts* at SOTA. Then, they give you the measurements so that YOU can decide whether you want to laugh and mock or not.

When I see new high end speakers from a well established company announced, or that are getting some buzz, there's a pretty good chance I'm going to see those speakers reviewed and measured by Stereophile. The chance they'll be reviewed at ASR is, at this point, close to nil. (Again, as much as the work Amir is doing is wonderful with what people are willing to send him).

So, in a nutshell: Don't Throw The Baby Out With Your Bathwater!

Even if the measurement device used at ASR may be more comprehensive than Stereophile's, Stereophile's measurements have been referenced as pretty good at capturing much of what you want to know or look for, and the product reach has been and will likely continue to be far wider (in terms of small and big name manufacturers, and many more larger speakers being measured).

So, Stereophile has been serving up a combo plate for a long time: the subjective review portion is there for those who like and value such things; the measurements are there if you only care about those. And of course, many people like myself enjoy reading both.
(But, since you don't apparently, you want what others may be enjoying to die). For these reasons, it seems rather selfish to want the magazine to die, as you seem to gleefully desire, but also foolhardy to think ASR will kill off the magazine. You think people needed ASR's measurements to question subjective reviews? Stereophile ITSELF has been providing that ammunition in it's own measurements section for decades, where you see the subjective description and the measurements to compare!

There's room for all, ASR measuring and reviewing whatever someone will send them, and Stereophile with it's greater reach in to the current high end landscape of products, with it's subjective component that many will continue to enjoy.

reponkic's picture

... can be found in the Stereophile's "best annual" amplifiers. Compare their prices with their distortion numbers -- results are clear, way too many products that receive glowing reviews by Stereophile (and a bunch of other audiophile publications) have subpar distortion and/or noise numbers. There is little to no link between price and performance in the audiophile world, but tons of snobbery, science denialism and, of course, overpriced trinkets for sad fantasists.

But I have to agree with you that there is a function for such publications -- to help separate snobs and science deniers from their money. Anyone, who pays tens of thousands for a tube amplifier, when he could have a Benchmark AHB2 for $3k, deserves to be fleeced.

supamark's picture

It's always sad when someone is so self centered that they think their subjective tastes are somehow objective reality for other people. They're not. Personally, I'd take a MC275 over 2 AHB2's any day of the week, and twice on Sunday but would never be dumb enough to suggest that either is objectively "better" than the other.

Glotz's picture

I own Benchmark products, and I bought them for their great sound!

I stopped wacking off at the measurements on the 2nd day of ownership.

What Benchmark equipment do YOU own?

It would appear that you haven't listened to a lot of expensive equipment in your experiences, but perhaps just measured? Care to share?

(Just trolling the troller... lol) You rather sound like a sad listener that doesn't own ANY great sounding gear... and hates others for doing so.

I don't think you're helping anyone save money whatsoever, but rather exhibit jealousy and find rationalizations for your behavior.

Robin Landseadel's picture

I'd like to note that Amirm has been buying some of the gear, including speakers, reviewed in ASR, with some of the funds sent to the site. Also, the most recent speaker review, of the Revel M55XC outdoor speakers [bought by an ASR member] gets into an issue with Amrim's technique for frequency response measurements as the speakers are intended to be used nearly flush against a wall. The measurements indicate a bass suck-out that would be filled in with wall placement. Amrim happens to think these speakers are excellent for the intended use, though it's safe to note that outdoor speakers usually are nothing to get excited about.

https://www.audiosciencereview.com/forum/index.php?threads/revel-m55xc-outdoor-speaker-review.14881/

RH's picture

"Compare their prices with their distortion numbers -- results are clear, way too many products that receive glowing reviews by Stereophile (and a bunch of other audiophile publications) have subpar distortion and/or noise numbers."

And where did you get the information about their "distortion numbers?"

That's right: Stereophile. You wouldn't have known that information about the specific gear if Stereophile hadn't measured that gear for you. So your own logic is self-defeating in terms of Stereophile being useless, or dead, or something we should want to see go away.

"but tons of snobbery, science denialism and, of course, overpriced trinkets for sad fantasists."

Again? You accuse others of "snobbery" in the same sentence you dismiss them with an air of superiority as "sad fantasists?"
Ironically you are simultaneously helping yourself to the information gained from Stereophile to act like a snob and dismiss the magazine and it's readers.

"But I have to agree with you that there is a function for such publications -- to help separate snobs and science deniers from their money."

And to provide YOU with the objective information you are using to understand which pieces are poor performers, right?????
You must be reading at least some of Stereophile's reviews, but apparently you are the only smart one in the room who can read it sensibly? Try giving other people a bit of credit.

Try for a moment to think outside your own point of view to see how the magazine may have value to others. I'm far from alone in enjoying reading the subjective part of the reviews while also reading the measurements to gain the more objective info. Then...like a Big Boy, I and others can make my own decision advisedly. It's when you filter everything, every other person's choices, through your own goals, likes and dislikes, that you can adjudicate everyone else as irrational or naive children. And you would be happy if something that provides plenty of value, entertainment and satisfaction to others would go away.

That train of thought is a deeper problem than anything you are citing about Stereophile.

"Anyone, who pays tens of thousands for a tube amplifier, when he could have a Benchmark AHB2 for $3k, deserves to be fleeced."

Sure, if you just ignore every reason someone else may have for choosing something different than you. I own a Benchmark DAC. I could easily own a Benchmark amp. But I own what were expensive at the time tube mono block amplifiers. Why? I prefer the slight distortion profile from the amps. I find it pleasant. You may not, but I do. I like how tube amps look. I like them conceptually. I like that the glowing tubes, which I find beautiful, are actually the music itself traveling through those tubes. I like the long heritage aspect of tube amplification. It's like many of the reasons mechanical watches still have a market: yeah you can get more accurate performance in a vastly cheaper digital time piece.
But the digital watch does NOT provide the aesthetics and the conceptual pleasure many get from mechanical watches.
Since people VALUE those things, it is completely rational to buy a mechanical watch. Since I VALUE the aspects of a tube amp that are different from the typical solid state amp, it's rational for me to acquire the tube amp.

If you can't understand how things you don't happen to value can have value for others, which in turn makes their own choices reasonable, then you'll continue to be befuddled by the world and judge everyone who thinks otherwise as inferior.

It's ok if someone reads Stereophile and buys a tube amp. It really is. The objective info is usually right there so anyone who values objective information can use it.

reponkic's picture

1. If I want "tube distortion" (or any other kind of distortion), I can program a little digital filter and run my music through it for FREE! It is also much more flexible than your distorting tube amp and fun to do. But I guess, you don't feel like programming and rather waste big $$$ on an obsolete machine to achieve the same result.

2. And BTW, what is "self-defeating in my logic"? Nothing! What is self-defeating, is Stereophile undoing their subjective reviews (i.e. nonsensical postmodern verbiage) with published measurements.

3. About mechanical watches -- snobs buy mechanical watches because of conspicuous consumption, but know precious little about them, usually only that "they don't need batteries". Sad clowns of that sort buy Submariners or Daytonas (sometime Offshores) because they want to display wealth and saw magazine adds to support that view. Very few people understand watches and mechanical movements; they deserve respect, and they are typically not snobs.

RH's picture

"1. If I want "tube distortion" (or any other kind of distortion), I can program a little digital filter and run my music through it for FREE!"

I work in digital sound. I manipulate it all day long with filters and effects. That there are tube emulation filters is not news to me. Again: consider other audiophiles who buy things you wouldn't buy may not be so naive as you assume.

"It is also much more flexible than your distorting tube amp and fun to do."

What you find fun may not be fun to someone else. Someone else may find owning a tube amp more fun than learning code or buying a digital box. Some people find listening to vinyl to be fun (I do). Some people find it to be a useless hassle. People are different; they like different things. Your "fun" is no more valorous than mine.

"But I guess, you don't feel like programming and rather waste big $$$ on an obsolete machine to achieve the same result."

No, I don't feel like programming a tube distortion filter for my stereo system. I frankly like to take a break from all the kind of stuff when I go to my stereo. And when you say I "waste" money on an obsolete machine to achieve the same result, you have of course completely ignored all the other reasons I gave for why the tube amp brings me joy.

"About mechanical watches -- snobs buy mechanical watches"......

"Sad clowns of that sort buy Submariners or Daytonas (sometime Offshores)"

Ok...you aren't changing your stripes any time soon about this.
You are utterly failing to consider other goals outside your own. This is a problem. You will remain befuddled about other people so long as you refuse to consider their actual motivations and reasoning, rather than belittle them with shallow pop psychology to make yourself feel superior.

It's too bad. This is the kind of imperious mocking that gives "science-oriented" audiophiles a bad name, and is counter-productive to spreading that concept. Fortunately there are more nuanced views out there as well.

Jim Austin's picture

It's too bad. This is the kind of imperious mocking that gives "science-oriented" audiophiles a bad name, and is counter-productive to spreading that concept. Fortunately there are more nuanced views out there as well.

Hear, hear!

I would add to what you've written that it really doesn't matter whether our friend understands your motives, or mine, or anyone else's. You are kind to humor him with explanations, but it's quite unnecessary. He should recognize that people have diverse views and desires that are just as legitimate as his own, and respect them, because people are worthy of respect until they prove otherwise--that's basic human decency. Most people--including most scientists--understand that. Apparently, he doesn't.

Jim Austin, Editor
Stereophile

reponkic's picture

Jim, everyone understand your motives. You want to sell your publication, and "dry" measurements don't sell nearly as well as subjective "reviews". It is perfectly fine, though not my cup of tea. As far as our friend's motives are concerned, I don't really care. He seems to be tired of audio science and needs a distraction.

What is disturbing though, is your claim that all views are legitimate. Next time when you are around an anti vaxxer or a climate change denier, remember what you wrote -- and don't forget to respect their views. :))

Jim Austin's picture

That's a common error among your type. Enjoying the audio components you like best is more akin to reading poetry than to denying the holocaust. Even if you assume that every aspect of it is a lie--and of course it isn't--it's still harmless. No children or animals are involved--only consenting adults, and no one's health is at risk. You're attacking people for choosing their own harmless pleasures.

Jim Austin, Editor
Stereophile

Pages

X