As We See It

Us and Them

Steve Guttenberg  |  Mar 16, 2015  |  First Published: Apr 1, 2015

I've known a lot of folks with impressive LP and CD collections who were perfectly content with the sound of the crappiest of hi-fis. This diverse group has included recording engineers, musicians, and owners of record stores. Loving music isn't the same thing as caring about the sound of music, and maybe, in some alternate universe, those folks would-have-been, could-have-been audiophiles. But in this universe, they didn't, and I'm not sure why.

It's clear that we audiophiles are a different breed. We crave a deeper connection with the sound of music. We also love gear, so we're much more likely to do whatever it takes to accommodate speakers and electronics that take up an inordinate amount of our living space. To us, the gear looks cool. "Civilians" think we're nuts.

My sound obsession began in the 1950s, when I was a small child frequently found pressing my Sony six-transistor AM radio to my ear. Ah, there's Jerry Lee Lewis singing "Breathless"—or Chuck Berry's "Maybellene." I was always searching for that magic mojo.

Many audiophiles credit early exposure to the audio system of a dad, a relative, or a neighbor as the spark that ignited their passion for sound. My family didn't have a hi-fi, but in 1961 we bought a Wurlitzer jukebox, a refrigerator-sized thing decked out with garishly colored lights that took over the basement den. We'd load it up with 45s and play tunes. The Wurlitzer was my dream machine. There was something about the way its disc-changing mechanism plucked the selected 45 from the rack and the massive tonearm ever-so-gently lowered the stylus into the lead-in groove that I found endlessly fascinating. The whirs and clunks of the machine going through its motions were all part of the intrigue. I spent hours sprawled on the checkerboard linoleum tile in front of the Wurlitzer, feeling the beat, massaged by the low, churning rumble of its mighty woofers.

My pipsqueak six-transistor radio didn't stand a chance. It played the same lyrics, melodies, harmonies, and rhythms, but through the Wurlitzer the sound was completely different—and a whole lot better. I never got over that. I'm still searching for better sound today.

My best friend's dad had a big-rig system that included McIntosh electronics. I can't remember the rest of the gear, but whatever it was, it was impressive, and the Macs' tubes put on a hell of a light show. To me, they looked like rocket ships. My friend's folks didn't play rock'n'roll, just classical music, and I don't recall much about the sound itself—it was background, not foreground, and so no threat to the mighty Wurlitzer.

I began to realize that not all music systems were equally engaging. I was into James Brown, girl groups, and Thelonious Monk, but by the time the Beatles hit I had my own little stereo system with a Garrard turntable, a plastic amp, and XAM speakers. The sound slammed into high gear when I discovered Jimi Hendrix and Led Zeppelin. Miles Davis's Bitches Brew came later, and a little further down the road were Brian Eno's Here Come the Warm Jets, Another Green World, and Before and After Science. Those were the albums I'd play when I checked out new gear. The better the hi-fi, the more of the sound I was privy to. The metamorphosis from music lover to audiophile was complete, and there was no turning back.

It wasn't long after that I began a 16-year stint of selling high-end audio in New York City. I demonstrated great-sounding systems to thousands of non-audiophiles, most of whom were unmoved. They didn't get it. Wilson Audio speakers and Krell electronics, or Quad ESL-63s with Conrad-Johnson amps, failed to get a rise out of most high-end virgins. Worse, they looked disappointed—the visual impact of the gear didn't seem to have anything to do with the sound they heard. I learned that exposure to great sound rarely produces instant converts, so it was especially satisfying when a non-audiophile was turned on by the sound.

In the early 1980s, when folks en masse were dumping their LPs for CDs, I began running comparisons of the formats for walk-in customers. Few non-audiophiles had ever heard a great high-end turntable, and through those demos I sold 'tables to folks who were blown away by the difference they heard—and who then began to really listen. The ones who weren't swayed were much more likely to go for the smallest possible speakers and fuss-free operation. That was all they needed—for them, a hi-fi was an appliance.

One of the things that first attracted me to high-end audio was its human scale. No one knows or cares who designed their iPhone or Samsung flat-screen TV, but owners of Rogue Audio amps can pick up the phone and chat with Mark O'Brien, the man who designs them—or ask John Grado about their 20-year-old Grado Reference cartridge. These guys actually like to hear from their customers. High-end audio companies are for-profit businesses, but the most successful of them are still run by people who put their blood, sweat, and tears into making the best-sounding products they can. That, more than money, is what drives them. It's all about the sound of the music.

No one needs a high-end system to play tunes, but it sure as hell is a better way to emotionally connect with music. Why we can't be satisfied with the "rich" sound of a Bose SoundLink Bluetooth III speaker is a small mystery. I dunno. Those pint-sized things may be good enough for most music lovers, but audiophiles want more than good enough—a lot more.—Steve Guttenberg

COMMENTS
Osgood Crinkly III's picture
Submitted by Osgood Crinkly III on March 16, 2015 - 4:38pm

I'm somewhere in between. I have a gargantuan "hi-end" system, on which I listen to electronically reprocessed stereo Chess recordings. Yes, it's wonderful to have great sound, but it's just icing on the cake. The most important thing is the music, not the makeup.

Litmus test is the awful taste of hi-fi mags. Have wasted hundreds of dollars on their recommended recordings, putrid Lyritas, Reference Recordings, Mercury's, Sheffields and RCAs. Have learned, by experience, to ignore music reviews of these tools.

When these "hi-fi" tools learn to listen to music, not sound, things might change. Most importantly, why are they deaf to harmonics? Seems engineers and hi-fi nerds have no idea of what they are. It's like when they look at paintings, they're oblivious to color.

James.Seeds's picture
Submitted by James.Seeds on March 18, 2015 - 4:40pm

I had one of those moments the first time around when a salesperson demoed a Krell and B&W (happy) Matrix speakers I caught the bug then, 10 years later I re-caught the bug with an all Meridian system, the sound was so overwhelmingly great that I was depressed after that audition

doak's picture
Submitted by doak on March 16, 2015 - 6:46pm

...of myself for resisting temptation to write a snarky joke about the "BMW speakers" referenced in the previous post.

dalethorn's picture
Submitted by dalethorn on March 16, 2015 - 9:23pm

The music is a given - whatever you have - LP, tape, CD, download - get the best you can. Once that bird is in the hand you can go for the birds in the bush. Long ago I hacked a lot - replaced cartridges in low-cost phonos, replaced speaker drivers and experimented with stuffing of enclosures - many times those changes made tones audible that weren't there before. Olson Electronics and a few others were my friends. In the 1970's I scraped together a few dollars on my meager pay as a warehouse worker to buy an AR turntable, Grado cartridge, Dynaco preamp, Crown DC60 amp, Advent speakers, Beyer headphones. Eventually some Rogers LS3/5a's, Fulton FMI80's, Dahlquists, etc. Advent 201 cassette deck, Akai and Sony reel-to-reel, ....

Now if I had more money I might have gotten Klipschorns, Fulton J-Mods, or HQD's. I might have purchased a Nagra tape recorder and Audio Research amps. But between what I had and a little mod'ing here and there, I got a LOT of musical enjoyment out of that gear, most of it from Recommended Components.

So is it about the music and not the gear? That's like saying you can have a taste of a fine salmon fillet, but just a small taste, and then back to fish sandwiches at McD's. I avoid those sandwiches for healthier foods. Same with audio - my understanding of orchestral music and tonalities of the various instruments has improved greatly with gear that can reproduce those sounds with reasonable accuracy. There may be a point in hi-fi gear where spending twice as much gets you basically the same thing as buying a car that costs twice as much but has no better functionality. However, just on an instinctive level I can see that many if not most of the bigger spenders on hi-fi gear are after greater functionality.

bernardperu's picture
Submitted by bernardperu on March 16, 2015 - 10:22pm

I Did not quite understand the point of the article. Or maybe i got it, but i felt like i was reading mp3.

All i kept wondering was...what do we audiophile readers want? Do we want a carefully crafted thought provoking article that reminds us of a great recording pressed on vinyl or a quickly written superficial rambling rant that sounds like loud mp3 with squashed dynamics?

I miss your vinyl style of writing, steve. I know it takes more time, you have to dig deeper into your feelings and the textures of your imagination, ignite your own creativity, and possibly make less money; but honestly, screw loud mp3s! You have become a 21st century pop writer.

lo fi's picture
Submitted by lo fi on March 17, 2015 - 4:43am

Reading this reminds me of so many articles that I've read before here and at other audiophile publications. Though I don't doubt the author's sincerity, there's a tired predictability to this nostalgic trip that he takes us on - the kindling of his interest in audio at an early age and the inevitable reaffirmation that audiophiles are unique and special regardless of how the mainstream perceives them. I can't help feeling that it's symptomatic of a pursuit where anything new and insightful to say was said long ago.

aaybes's picture
Submitted by aaybes on March 17, 2015 - 4:41am

Lately, between the Pono debate, the WSJ, Gizmodo, The Verge article responses, stereophile and its sister publications seem to have become overly defensive.

Nothing good ever comes out of the "us vs them" rhetoric. All this does is pit people against one another instead of re-uniting. Everyone who loves, or even just merely enjoys music, wants to connect with music.

Audiophiles, and I will admit to really disliking this word because in and of itself it categorizes people, ranks people even, enjoy the somewhat futile pursuit of pure sound, but that's a hobby, that's something they like doing. I too, have spent an amount of money on audio gear that most people would think is ridiculous, I too, read the pages of audiostream, strereophile, innerfidelity and a number of other publications, reading reviews and articles about awesome audio gear, new tech, new albums, but when I come across this type of articles, I get sad.

This type of post just divides people more, if you believe in what you write about, if you believe in music, you should write about what will bring people together, you should write about amazing new gear, about how real HD audio files are better, you should recommend proper HD recordings, you shouldn't dismiss the others, "them", as people who don't care to connect to their music at a level as deep as "us", the true music lovers, the "audiophiles".

Alex.

anthony.aaron47's picture
Submitted by anthony.aaron47 on March 17, 2015 - 10:11am

I purchased a new system in April, 1986: KEF 105.2 speakers (kind of like B&W 801 of the day, but more affordable), Well-Tempered Labs turntable and arm, Koetsu MC cartridge, and all Nakamichi electronics (PA-7, CA-5, OMS-7, etc.) at a cost of about $13K (equal to about $50K in today's money - likely more). It was an excellent system -- in fact, my former wife still has it, and it sounds great.

I, however, had a different economic future after the divorce than she did -- and it is only within the past 18 months that I was able to purchase a CD-based system for myself: Omega Super 3S speakers (sadly, no longer made), Dared 2A3C integrated amp (3.5 wpc) and a Marantz CD player. Total cost, less than $3K.

Once the speakers and amp broke in (about 100 hours, give or take), the magic of this simple system became obvious. True, CDs are NOT my favorite source material, but I do not have the WTL equipment or the 1300+ LPs that I once had -- but, in making the best of a CD-based system, I am very pleased with the quality of the sound.

I guess that my point is that it's possible to have a high quality, high-fidelity system for less than the cost of a custom home. True, I'd love to have my old turntable and LPs, but financial constraints just won't permit that. I'm totally satisfied with what I have - and I love the sound of the music that comes from this simple system - it truly enriches my life like few other man-made phenomena can.

mauidj's picture
Submitted by mauidj on March 17, 2015 - 10:54am

This article is a perfect illustration of why I have stopped reading Audio magazines. (After over 30 years as a Stereophile subscriber)

Now we get magazines full of stuff about the the writer or the industry or some other esoteric ramblings. And we won't even go into how defensive and antagonistic they are towards their readers. Alex's post hits the nail squarely. It's all about Them and Us and what is so amazing is that the "them" is often your own readers!!!!

Want nostalgia? How about some decent articles about HiFi? About how to make your system sound better...and I don't mean by adding some stupid Synergistic Research type box of bits or a fucking brass gong!!!!. No, some real hobbyist stuff. Speaker positioning. Grounding your system. Tweaks that work and don't cost a months paycheck? Buying used gear. How to audition a component. Etc etc.

Oh..and how about not getting on the back of, or deleting the post of, anyone who would dare to call out a writer over some stupid Synergistic Research type box of bits. Not to mention the very public rants by a certain editor in the general media when discussing the recent vinyl fashion craze. Talk about them and us! The industry does not need your "protection".

Gone are the days of Scull et al and their wonderful, funny and genuinely useful articles. Stereophile and the others have degraded into a never ending catalog of audio show "reviews" and self promoting ramblings.

Good luck saving the industry from extinction with these methods.

bernardperu's picture
Submitted by bernardperu on March 17, 2015 - 7:57pm

You are a bit harsh, but I believe you hit many nails. The HiFi media needs better articles. An audiphile is first an foremost a conscious listener, not a gear collector. Music quality and our mental ability to enjoy it to a a full degree come first.

Blind tests are a must (Founder of Sterophile, rest in peace, totally agrees).

Well crafted articles that get us closer to the music are necessary.

Raise the price of your magazine and start depending on readers, not advertisers, especially if they advertise products that are enemies of blind tests (cables and gear that appeal to the biases of the mind).

dalethorn's picture
Submitted by dalethorn on March 18, 2015 - 9:52pm

Blind tests may work when the test subject has total control of all aspects of the test except one: knowing which is which at any given moment. But when blind tests have the kinds of flaws noted in the following article, they become useless.

http://www.stereophile.com/content/listening-143

bernardperu's picture
Submitted by bernardperu on March 18, 2015 - 10:23pm

Agreed. For audio equipment, long and comfortable blind tests are feasible with the assistance of a high school or college student on minimum wage.

The most important point is that findings of subtle differences are unreliable unless a blind test is used. Cable or other esotheric gear reviews should not be published unless blind tests are used. Highly regarded manufacturers of hi fi are non-believers of cables. Their opinions should be accounted for and disputed only by using the minimum standard of blind tests for cables.

dalethorn's picture
Submitted by dalethorn on March 19, 2015 - 5:23am

The thing I suspect that you might be missing is that different cables are likely to sound a little different at least. Subtle? Yes. Unreliable? Possibly. But not because the differences aren't repeatable or consistent. The problem is judgement, just like the article I referenced described. Who among the blind testers is going to know, when a difference in a blind test is very subtle, what the essence and meaning of that difference is? Give that some thought. Like the art expert, the audio expert has to report the subtle difference from the perspective of his experience. He could say "A is a little brighter", but when you think of all the things going on in a complex audio system, that tidbit isn't helpful. He needs to know if it's brighter 'good' or 'bad', and that requires a lot of experimenting. I'm sure that someone will argue that blind testing should accompany each and every step of that process, but even if it were theoretically possible to do so, the article I referenced on art testing should help clarify why it just isn't feasible.

bernardperu's picture
Submitted by bernardperu on March 19, 2015 - 8:06am

The blind test environment should be the one the reviewer has always used.

I believe all the reasons brought up by you why a blind test is impossible would be solved by having a trained assistant on minimum wage, wouldn't they? In the end it is just a matter of changing cables, as the reviewer can change the set up himself.

Here are the major reasons why blind testing is a must:

1) All the inherent human biases one can think of. Golden ears have biases too, of course. Not using blind tests for subtle difference is just like asserting: "I am a computer."

2) Highly regarded manufacturers use blind tests for their own products. One huge example: Magnepan.

3) Many highly regarded manufacturers do not believe in expensive cables. I have confirmation from some (I own their products): Schiit; BAT; Sanders. http://www.roger-russell.com/ (former director of acoustic research at mcintosh)

4) Conflict of interest. Cable manufacturers are big advertisers (they even hire reviewers and offer them more money). I believe reviewers rig the game so they don't have to consciously lie. They allow the biases of their mind to do the dirty work so they don't lose advertisers (and job opportunities). No blind tests = game is rigged.

Any of the above reasons would suffice in order to make blind tests mandatory or, else, do not publish the reviews.

Thanks for the exchange!

dalethorn's picture
Submitted by dalethorn on March 19, 2015 - 8:23am

Everything you said is technically correct, but it seems you are unwilling to read the article I pointed to, and understand what it says. I'll summarize: The real-world cost in time and money to perform hundreds of blind tests, which an expert reviewer can perform alone rather quickly, is prohibitive. Assume the audio expert works at the same rate as an average lawyer, i.e. $400/hour. If the reviewer can perform his tests in 3 days (24 work hours), the cost is $9600, or ten thousand dollars. What you're asking would cost at least one hundred thousand dollars. I've been there and done that personally, and the only feasible shortcuts are the expert's knowledge and experience. You should consider that the reason you can't get the Blind Test Proof that you want is because it isn't doable. If you still feel it is doable, find someone who's done it and fully documented every little step. In software development, I've worked closely with QA on a number of projects, and where some companies consider that closeness to be a potential conflict of interest, they have to pay heavily for the non-cooperative testing, and also accept an inferior product in most cases.

bernardperu's picture
Submitted by bernardperu on March 19, 2015 - 10:42am

Why are blind tests expensive? Not at all.

Assistant on minimum wage should cost USD$200 at the most per review. In many cases, assistants are not necessary, especially if a headphone rig is used. Reviewer can blindfold himself and switch the cables blindfolded (blind people do far harder tasks).

Not sure how Rob Watts (maker of Chord Hugo) conducts his blind tests, but my guess is that he does not use an assistant, just a headphone rig and enjoys the fun of switching cables and equipment while blindfolded. The only thing I know for a fact is that Rob Watts considers blind tests essential.

If these blind tests are not feasible, then, reviews on cables or other esoteric gear should not be published as "reviews."

I have not read the article you have referred to. I do not read millions of articles. Unfortunately (or fortunately) I am forced to carefully curate how I assess my "learning time." One of the constraints of being human: limited time. The reasons why I have not read the article is because I am highly likely to find flawed arguments that cleverly support personal financial interests. Points I am not likely to find in the article:

1) The inherent biases of the human mind and how this makes blind tests mandatory (read Rob Watts on this).

2) The conflict of interest at stake and how this clearly will affect the human bias.

3) The inclusion of the opinions and findings of highly regarded manufacturers. These opinions are systematically ignored by reviewers. So is Gordon Holt's highly emphatic opinion on the matter.

4) The financial viability of comfortable and long blind tests by using an assistant on minimum wage.

5) Blind tests on a headphone rig without an assistant.

dalethorn's picture
Submitted by dalethorn on March 19, 2015 - 11:06am

So you think a minimum-wage assistant is reliable? You should be reading toothpaste reviews, not complex expensive audiophile gear reviews. You continue here to deny that which you've refused to even read, i.e. the article explaining *exactly* why totally blind tests of the aforementioned complex gear cannot work. The arguments you make (repeatedly!) I've already addressed, by acknowledging that a good reviewer has to be objective and know how and where to apply "blinders", which is an important skill for reviewers. You're trying to reduce the judgement factors to pure objective facts, which is very much the same as saying that a reviewer's skill and judgement is not important.

So now that you've explained in detail how you disagree with the reviewers here insofar as their reliance on personal judgement, you're free to go find some of those pure blind testing reviews and make your purchases accordingly. And BTW, J. Gordon Holt made it abundantly clear that his ears trumped all "objective" tests performed by his competitors in his day. Stereophile's very reputation was founded on listening evaluation, with measurements secondary, as confirmation *only* when the testing tech was sufficiently advanced and available for that purpose.

bernardperu's picture
Submitted by bernardperu on March 19, 2015 - 11:23am

Year 2007 http://www.stereophile.com/asweseeit/1107awsi/

"John Atkinson: Do you see any signs of future vitality in high-end audio?

Gordon Holt: Vitality? Don't make me laugh. Audio as a hobby is dying, largely by its own hand. As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me, because I am associated by so many people with the mess my disciples made of spreading my gospel. For the record: I never, ever claimed that measurements don't matter. What I said (and very often, at that) was, they don't always tell the whole story. Not quite the same thing.

Remember those loudspeaker shoot-outs we used to have during our annual writer gatherings in Santa Fe? The frequent occasions when various reviewers would repeatedly choose the same loudspeaker as their favorite (or least-favorite) model? That was all the proof needed that [blind] testing does work, aside from the fact that it's (still) the only honest kind. It also suggested that simple ear training, with DBT confirmation, could have built the kind of listening confidence among talented reviewers that might have made a world of difference in the outcome of high-end audio."

Can you kindly apologize for being a cyber bully?

dalethorn's picture
Submitted by dalethorn on March 19, 2015 - 11:29am

Your Cyber Bully comment highlights your apparent dishonesty here. Your agenda is becoming clear - your many and lengthy posts should indicate that you're very serious about blind testing, but then you say that your time is too limited to read the most essential counter-argument to blind testing - the short article I pointed to.

So which is it? Are you serious or not?

And BTW, your quotes from Holt only back up what I already said - that the test/measurements are for confirmation, not to say how a component sounds.

bernardperu's picture
Submitted by bernardperu on March 19, 2015 - 11:37am

Ok, I will read the article.

You are blindly skipping the most important points made by Holt:

Blind tests are a matter of basic honesty.

Not using blind tests is an embarrassment and it is not rational. Holt claims mocking those who refuse blind tests is acceptable.

dalethorn's picture
Submitted by dalethorn on March 19, 2015 - 12:51pm

Actually you're misrepresenting what I said, I think. I don't argue against blind tests absolutely - as my comments noted, a good reviewer will know where and when to put the blinders on, without the interference of a "helper" who is certain to be unreliable in every way. Remember that the reviewer needs to have total control of everything except the "switch", and that won't be possible when a human being who's not highly, highly motivated to get at the truth is holding that switch. If you want to hire a helper to hold that switch - don't - get someone to connect the equipment in such a way that how it's connected and to what is not visible to the reviewer, then the reviewer can hold the switch. All the assistant would then be needed for is to verify which component is 'A' and which is 'B'.

But let's dig a little deeper. If this blind test is in a home with no disturbances - just the reviewer and his assistant, how can you possibly trust that? I can think of a thousand ways that can be turned around to any result. I won't say any more on that. But going to the other scenario, of a room full of verifiers who have made absolutely certain that X is connected to Y and nothing whatever is hidden (except to the testers or reviewers) - the Stereophile article I pointed to details the folly of that approach.

So you'll have to explain how we can have a blind test, in a very quiet place with no pressure of any kind, yet fully trust the results. And assuming that the test(s) have many many exceptions and perturbations, how to accomodate each one economically and distinctly from each other.

ChrisS's picture
Submitted by ChrisS on March 19, 2015 - 11:39am

Also, according to John Atkinson, JGH never did DBT either.

dalethorn's picture
Submitted by dalethorn on March 19, 2015 - 1:03pm

I suspect that Holt evolved his opinions to some extent during his 30-plus years with Stereophile, especially as new technologies came along. I do think his basic premise was that measurements are good to quantify what we hear, but we shouldn't expect measurements to tell us exactly how something sounds, since the interactions between different components aren't fully predictable.

John Atkinson's picture
Submitted by John Atkinson on March 20, 2015 - 12:28pm
bernardperu wrote:
Gordon Holt [in 2007]: Audio as a hobby is dying, largely by its own hand. As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal.

You should note that despite his outspokenness on this subject in my 2007 interview with him, Gordon never performed blind testing to reach his review conclusions in Stereophile. In addition, almost all the blind tests in which he did take part after I took over from Gordon as Stereophile's editor in May 1986, were organized by me.

John Atkinson
Editor, Stereophile

John Atkinson's picture
Submitted by John Atkinson on March 20, 2015 - 12:23pm
dalethorn wrote:
The real-world cost in time and money to perform hundreds of blind tests, which an expert reviewer can perform alone rather quickly, is prohibitive.

Quite right, dalethorn. I have ceased to be surprised by people who tell me that blind tests are easy and cheap to perform. I took part (as a listener) in my first blind test in the spring of 1977 and since then have been involved in well over 100 such tests, as a listener, proctor, or organizer. As a result of that experience, I feel that such testing isn't feasible for a magazine with a small staff and a monthly publishing schedule.

Organizing a blind test that correctly identifies small, subtle but real differences is difficult, time-consuming, and expensive. And it really only produces valid results when you are testing something with a binary outcome: is there a difference, yes or no?

Let me give you an example. Five years ago, I visited B&O in Denmark. B&O may be a company that focuses on "lifestyle" products but they have a superbly equipped R&D department, with a staff of highly qualified engineers. They routinely perform blind tests, to determine, for example, if they should use a $5 capacitor at a critical point in a circuit or if they can get away with a 5-cent part.

Such testing involves trained listeners, each taking part one at a time in a session that lasts no longer than 45-60 minutes. These tests are repeated, sometimes for several weeks, using many different music examples, so that the end result is an enormous amount of data that can be analyzed with ANOVA statistical techniques.

That's how you design a test where there might be a real but small sonic difference. If you cut corners on designing or performing a test like this, you end up with meaningless results, just as has been demonstrated by so many blind tests of things like cables that have been performed by self-proclaimed skeptics.

John Atkinson
Editor, Stereophile

bernardperu's picture
Submitted by bernardperu on March 20, 2015 - 1:04pm

So how do Robb Watts, maker of Chord Hugo, or Magnepan do their blind tests? Have you asked them? Please don't tell me that these would be irrelevant questions or that they would not capture your readers. Why is the hifi media so afraid to ask the right questions regarding blind tests and esoteric cables? You interview manufacturers once in a while. You love their products. Why don't you ask them what they think? You will get lots of clicks.

I am audiophile who believes in subjective listening. But at the same time, just like you, I believe in objectively controlling variables whenever possible.

Long and comfortable blind tests are cheap. Would an assistant on minimum wage make you uncomfortable? I am sure the reviewer would hire someone he feels comfortable with. As easy as hiring a nanny.

So B&O does the best possible blind tests out there? Sure! But that does not remotely mean that that is the only way to do blind tests. Neither does it mean that corroborating your findings while being blindfolded does not beat your current system exposed to so many human biases. I personally did some free of any charge blind tests to demonstrate that I can tell the difference between MP3 and CD-quality. They were free and they worked beyond reasonable doubt.

As far as trusting the reviewer goes: I will personally trust the reviewer. Period. I have been doing so for years. If reviewer claims he is using blind tests, I will trust him, and so will everyone who is already reading the reviews. In the end, there will be more readers and the Hifi community will grow.

Furthermore, no assistant is necessary if using a headphone rig. You can switch cables while remaining blindfolded. It also sounds like fun.

I really love your reviews and your articles, John. You obviously have lots of training as a sound pro and in the Liberal Arts. Will keep reading them, even if we have a small disagreement.

Why not follow Holt's advice and either incorporate blind tests for subtle differences or just not post the review altogether? I particularly mean cable reviews. If your review system is not good enough to reach reliable conclusions, why publish the review?

Highly regarded manufacturers firmly believe many super expensive cables, if not all of them, are pure snake oil. Why should you continue to systematically ignore their opinions? If you believe they can build great speakers, amps, and DACs, then, you just dont trust their listening skills when they tell us that expensive cables are sneak oil? That just doesn't make sense. You either trust their listening skills or you don't. And if you do, then, you should not systematically ignore their opinions.

Just my two cents.

John Atkinson's picture
Submitted by John Atkinson on March 21, 2015 - 9:44am
bernardperu wrote:
So how do Robb Watts, maker of Chord Hugo, or Magnepan do their blind tests?

If they wish to obtain reliable and repeatable results, they would perform such tests to a suitably rigorous standard. You can find a set of guidelines on how to design valid blind tests, ITU-BS.1116-3, at www.itu.int/rec/R-REC-BS.1116-3-201502-I/en.

bernardperu wrote:
Why is the hifi media so afraid to ask the right questions regarding blind tests and esoteric cables?

It's nothing to do with being afraid of the results. It is because, as I explained to you, performing blind tests that produce reliable and repeatable results when the sonic difference is small and subtle is time- and resource-consuming. Again please refer to the ITU guidelines lined above. And if you don't adhere to those guidelines and cut corners, you end up with results that are meaningless. That hasn't stopped dishonest and/or biased writers in the past from pronouncing that their blind tests "prove" there are no differences, of course, but those people are merely playing to the prejudices of those who already believe that there are no audible differences.

bernardperu wrote:
I am audiophile who believes in subjective listening. But at the same time, just like you, I believe in objectively controlling variables whenever possible. Long and comfortable blind tests are cheap.

Again, I must explain that my experience has been that performing such tests is neither cheap nor easy.

bernardperu wrote:
Would an assistant on minimum wage make you uncomfortable? I am sure the reviewer would hire someone he feels comfortable with. As easy as hiring a nanny.

Not at all. From the ITU guidelines I linked to above, in Section 3.1: "It is important that data from listening tests assessing small impairments in audio systems should come exclusively from subjects who have expertise in detecting these small impairments. The higher the quality reached by the systems to be tested, the more important it is to have expert listeners."

So you can't just hire someone off the street. You need to find one or more listeners who can demonstrate their ability to detect small but real differences, listeners who not only have demonstrably good hearing but more likely will need further training. The manufacturers I talk to who routinely perform blind testing start off with a large pool of potential listeners, who are subjected to basic tests of their ability to detect known audible differences. Most listeners are rejected, some due to hearing damage, some to the inability to reliably detect differences that prior work has shown to be audible. The manufacturer ends up with a small group of expert listeners to use for their blind tests.

Remember, they are not testing to see if the "man in the street" can detect a difference, they are trying to determine if there is an audible difference in absolute terms.

bernardperu wrote:
So B&O does the best possible blind tests out there? Sure! But that does not remotely mean that that is the only way to do blind tests.

Actually, that's exactly what it does mean if you wish to determine if a small, subtle difference is audible. Again, if you don't perform such tests with the appropriate rigor, your results are meaningless.

bernardperu wrote:
Neither does it mean that corroborating your findings while being blindfolded does not beat your current system exposed to so many human biases. I personally did some free of any charge blind tests to demonstrate that I can tell the difference between MP3 and CD-quality. They were free and they worked beyond reasonable doubt.

Except that such differences are not small, compared to those I experience with cables.

bernardperu wrote:
Why not follow Holt's advice and either incorporate blind tests for subtle differences or just not post the review altogether? I particularly mean cable reviews. If your review system is not good enough to reach reliable conclusions, why publish the review?

Again it is appropriate to point out that Gordon never followed his own advice in this respect.

Given that it is impractical for a review magazine to implement a properly designed blind test regime, there are two choices: use insufficiently rigorous blind testing, which will produce false negatives—ie, the results indicate that no audible difference exists when one actually does—or use sighted listening as introduced in this magazine by J. Gordon Holt more than half a century ago and accept that such a regime can produce false positives—not every apparently audible difference will be real.

Pareto's rule applies here: inadequate blind testing probably produces well more than 80% false negatives while sighted testing as practiced by experienced reviewers probably produces very much fewer than 20% of false positives. The choice to me is clear, and if you think that I should be condemned when I own up to the possibility of there being false positives in our review findings, we have always told Stereophile's readers that it is important for them to test those findings for themselves.

John Atkinson
Editor, Stereophile

bernardperu's picture
Submitted by bernardperu on March 21, 2015 - 10:14am

Thanks for your kind and extended reply, John.

There seems to be a misunderstanding regarding the use of an assistant on minimum wage. This assistant is not a listener. He/She would just switch the cables while the reviewer remains blindfolded. So that would be the only condition that changes in the way you make your current reviews.

While reading your reply I came up with an idea. In the near future, I will be starting an NGO that has blind people (particularly musicians and music enthusiasts) listen to music on my Hi-Fi. There is, of course, a lot more to this. One thing I could do is have them compare cables and document it. Not cables that have a small price difference between them, but big price differences. I have access to Audioquest cables. So they just sit and listen for hours in a comfortable manner. I will try to find out if they can reliably spot differences between a $50 and $1,000 pair of cables.

Unfortunately, the objectivists resort to pseudo science in order to establish their "scientific" conclusions. They don't even seem to engage in conscious listening. They believe science has found an explanation for everything, as if all cancers had been cured. Having said this, there are many real audiophiles among us (and excellent manufacturers too!) who believe cables don't make a difference that matches their business models. Let's use Audioquest as an example. A large number of subjectivist audiophiles and manufacturers do not believe Audioquest's business model reflect a nearly accurare return on cable investment. This is a biggie as Audioquest is a major player in the industry. My advice: Start asking manufacturers and see their answers for yourself.

BEst,

Bernard

ChrisS's picture
Submitted by ChrisS on March 21, 2015 - 11:32am

Having someone else listen to my stereo system will have nothing to do with whether or not I can hear what that other person(s) hears. They will also not decide for me whether or not I want to spend $50 or $1000 on a cable or any other component.

Reviews such as those published in Stereophile give me a place to start in my selection of cables (or any other component) and an idea of what to listen for. I've been shopping at a local brick and mortar shop for decades and they will allow me to borrow components over a weekend to try out in my system. I can usually hear a difference within minutes, if not by the end of an hour, of most components and by the end of the weekend, I can decide whether this new component is a worthy upgrade or not. I don't see the point of having a panel of blind musicians working with an underpaid assistant telling me otherwise.

Other than those shopping for handmade, artisan-crafted products, I don't know of anyone who shops by talking to the manufacturer. You try something and if you like it, you buy it. Seems simple enough.

By the way, I've been very happy with every Audioquest product I've purchased so far.

bernardperu's picture
Submitted by bernardperu on March 21, 2015 - 12:23pm

I believe you are not being only judgmental, but also addressing a whole different subject.

You are free to spend your disposable cash as you please. That is your right as a consumer and that is the point you are addressing. That is fine with me.

The point I am addressing has to do with the intellectual honesty of properly done research. If there is a big factor invalidating a particular research (in this case cable reviews), then, the research is simply invalid and, potentially, intellectually dishonest.

I have addressed at least 4 big factors that make cable reviews as published by stereophile potentally invalid. I am just criticizing the methodology used, the conflict of interest, and the blatant and systematic disregard for the opinions, listening skills, and methodologies used by highly regarded manufacturers.

And, finally, to be honest, I don't have any valid conclusions that would allow me to assert that Audioquest's business model is flawed. My educated guess suggests it is flawed, but I cannot assert it. All I am requesting is a more widely accepted way to conduct research/reviews. After the proper methodology is used, then, we can finally reach valid conclusions.

The proper methodology and approach to reviewing cables should include: 1) blind tests 2) the opinions of manufacturers. That latter would address a larger context.

So, let's say, a reviewer claims X cable is superior to Y cable and used Magnepan speakers and Pass Labs gear in order to evaluate the cables. Then, the reviewer should contact Magnepan and Pass Labs for their opinion. At least ask them if in their experience they have been able to find sonic superiority in any super expensive cables. The kind of questions asked to manufacturers would vary based on the initial conclusions by the reviewer.

ChrisS's picture
Submitted by ChrisS on March 21, 2015 - 1:57pm

Bernard,
You are devoting much, too much concern and energy towards what most would consider "marketing". With the help of discerning information from the likes of Stereophile, most consumers would find it easy enough to decide whether to spend $125 on a "most-bang-for-the-buck" cable or $12,500 on the "ultimate" cable.

If you wish to pursue "honesty" in products that have literally many millions times more impact and life-and-death consequences than consumer audio, then look no further than the tobacco, automotive and pharmaceutical industries.

Honestly.

ChrisS's picture
Submitted by ChrisS on March 21, 2015 - 2:17pm

How can you imply that what the reviewers of Stereophile do is "dishonest" and that the opinions of manufacturers are more "valid" when they have a vested interest in promoting their own products?

(Think again "tobacco", "automotive", "pharmaceuticals", etc.)

bernardperu's picture
Submitted by bernardperu on March 21, 2015 - 5:28pm

You have misinterpreted me. I don't think stereophile's writers are dishonest. I have purchased many items based on their reviews and i am happy.

ChrisS's picture
Submitted by ChrisS on March 21, 2015 - 9:56pm

Then why did you title your previous post "Intellectual honesty"?

Judging by the number of subscribers to Stereophile's paper and electronic magazines, one might think that the way Stereophile reviews audio components is widely accepted. I cannot say that I know anyone doing what you propose Stereophile should do.

dalethorn's picture
Submitted by dalethorn on March 21, 2015 - 9:47pm

I don't expect anyone to be honest, completely honest, or honest to the degree someone might require. I expect them to give me information, and I'm not going to nag them about the quality of their information. I've had discussions with people about 'facts', where the other parties to these discussions believed that a 'fact' was something proven indisputably. What can anyone prove, other than mathematical truth tables or simple geometric measurements? Facts to me are names, dates, places, events etc. If they're true or false and of sufficient importance, the truth to whatever extent will come out. What do people think when they read measurements of two or more samples of a speaker system or headphone, especially when the production of those samples is months apart, and the measurements are significantly different? Particularly when they purchased sample A and sample B measures better? This business of making and selling higher quality gear is very fluid, and I'm sure you recall reading warranties that contain "specifications are subject to change" etc. Gordon Holt sometimes worked with manufacturers of esoteric gear, Irving Fried being one example, where the manufacturer made several changes to improve the product when Holt criticized it. Holt sometimes became frustrated with those many changes, feeling that he could never get a 'final' review, and even if he did, the product might change right after the review anyway. But back to cables. Let's say that perfect blind tests were performed comparing cable A to B, stating that A outperformed B between source E, amp F, and speaker G. Will that tell me how cable A is going to sound on my system, or even whether A will still be better than B on my system? Would it surprise anyone to discover that sometimes the performance reverses on different systems?

John Atkinson's picture
Submitted by John Atkinson on March 22, 2015 - 5:06am
dalethorn wrote:
Gordon Holt sometimes worked with manufacturers of esoteric gear, Irving Fried being one example, where the manufacturer made several changes to improve the product when Holt criticized it. Holt sometimes became frustrated with those many changes, feeling that he could never get a 'final' review, and even if he did, the product might change right after the review anyway.

It was my experienece with a similar review of a Fried speaker in 1987 that led me to implement what we called in-house the "Bud Fried Rule": that when a manufacturer sends Stereophile a product for review, we review it "as is."

John Atkinson
Editor, Stereophile

bernardperu's picture
Submitted by bernardperu on March 23, 2015 - 9:04am

I basically agree with everything you said. I am postmodernist too! Also, I don't think that what you state above contradicts any of my premises.

I have learnt a lot from reviewers, especially the more skilled ones, like Atkinson or Guttenberg. But at this point, I feel like my learning process has taken me to listening to well educated, successful in getting us closer to the music, manufacturers.

Robb Watts, the maker of Hugo, is a wonderful example of all the great things about sound and music a manufacturer can embody.

I highly recommend you to read his posts http://www.head-fi.org/forums/posts/by_user/id/394072

Best,

Bernard

dalethorn's picture
Submitted by dalethorn on March 23, 2015 - 9:17am

Headfi is the best place to read Robb Watts' philosophy of equipment design? I would think that he has an official blog somewhere under his name.

bernardperu's picture
Submitted by bernardperu on March 23, 2015 - 9:49am

Apparently, head-fi is the main source for Watts. You will have to be patient in order to find those posts that are the most enlightening to you.

I have read all his posts up until 30 days ago.

Here are a couple of examples

blind tests:

http://www.head-fi.org/t/702787/chord-hugo/8610#post_10982025

USB cables

http://www.head-fi.org/t/702787/chord-hugo/8385#post_10924471

dalethorn's picture
Submitted by dalethorn on March 23, 2015 - 10:06am

You yourself said that your time is extremely limited, yet the first of your references goes to Page 575(!!) of the chat on that subject. Please explain.

bernardperu's picture
Submitted by bernardperu on March 23, 2015 - 10:26am

haha, my time is not limited. I have plenty of spare time. I was referring to the amount of research/knowledge existing in this world vs. the time of any human being. We are forced to choose.

Robb Watts is a great source, so I have allocated plenty of time learning from him. I rather spend a lifetime reading Nietzsche than a minute reading Coelho.

dalethorn's picture
Submitted by dalethorn on March 23, 2015 - 10:32am

The rest of us here do not have unlimited time to follow your links to a 600-page chat forum. Perhaps you could show some consideration and point the readers here instead to official documents, such as this:

http://chordelectronics.co.uk/files/Hugo-DAC-Technical-Master-Class.pptx

bernardperu's picture
Submitted by bernardperu on March 23, 2015 - 11:03am

just read Watts' posts, not all posts. Your call.

dalethorn's picture
Submitted by dalethorn on March 23, 2015 - 11:21am

I don't read headfi chat - just the ads. I avoid chat because it goes on endlessly, like certain posts here do sometimes. I'm quite serious about hi-fi, so I use my limited time to dig into material that contains principle as well as fact. Such as the articles I've pointed to here. So far you don't seem to have linked to any non-chat articles, but then I may have missed them.

dalethorn's picture
Submitted by dalethorn on March 23, 2015 - 11:36am

"...spend a lifetime reading Nietzsche than..."

"Once you get into him he's rather simple - childish almost." -- Star Trek, Where No Man Has Gone Before.

bernardperu's picture
Submitted by bernardperu on March 23, 2015 - 11:57am

Robb Watts's answers can be very enlightening and important because he addresses questions on blind tests and cables. Yes, it got me a while to get to the posts I wanted to read, but found info that taught me plenty within two hours. I don't usually read Head-Fi, but I certainly paid attention to Rob Watts.

Here are some links of more in-depth research you may enjoy:

For smart investment on speaker wire

http://www.roger-russell.com/wire/wire.htm

The science of how we humans enjoy music is pretty much non-existant. Listening matters the most.

http://arstechnica.com/science/2013/02/human-hearing-beats-sounds-uncertainty-limit-makes-mp3s-sound-worse/

http://phys.org/news/2013-02-human-fourier-uncertainty-principle.html

http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.110.044301

Evolution of composing music (especially pop music)

http://www.nature.com/srep/2012/120726/srep00521/full/srep00521.html

And a great piece by J. Atkinson where he uses a style I call "Applied Liberal Arts"

http://www.stereophile.com/content/2011-richard-c-heyser-memorial-lecture-where-did-negative-frequencies-go

dalethorn's picture
Submitted by dalethorn on March 23, 2015 - 12:49pm

I read Heyser as a member of the AES, and I've performed numerous speaker cable tests. If someone wants to follow your links, then bless their little hearts (as we say in the South). Otherwise, I'd suggest they skip the koolaid and do their own google searches from scratch.

ChrisS's picture
Submitted by ChrisS on March 23, 2015 - 4:31pm

Some of us prefer to listen rather than read about listening.

Oh heck, it's been quoted and paraphrased so many times, let's all join in on a rousing chorus of "Writing about music is like..."

doak's picture
Submitted by doak on March 17, 2015 - 8:21pm

Well said!!

Doak

blueingreen48's picture
Submitted by blueingreen48 on March 17, 2015 - 11:36am

I fell in love with music as a kid in the 50's listening to it on a cheap table radio and in the car. We didn't have a stereo but some of my friends' parents did, those classic old consoles complete with TV, that I thought had to be state of the art. Then came transistor radios and "suitcase" record players. I had one. I was in college before I heard really good speakers (a mono Klipschorn and that amazing KLH portable record player) and while I was blown away by it, it didn't spoil me for music heard on lesser equipment.

I now have good mid to low price equipment in my home and office. Someday I'd like to be able to afford what I hear in high end systems owned by people I know. But I'm not going to stop listening to and loving music while I wait.

deckeda's picture
Submitted by deckeda on March 17, 2015 - 6:43pm

Steve, we may crave a deeper connection to music than non-audiophiles, but musicians already have it. And so, a great hifi doesn't add more to the experience beyond a "yeah, that does sound really nice ..."

Read just about any of the old "Rick Visits" Stereophile features for more examples.

Musicians are already more in tune (no pun intended) with music through its creation and performance than anyone with a megabuck system will be. It's different, not "better" or "worse," so for the folks on both sides of the equation here, please don't get me wrong.

Johnny2Bad's picture
Submitted by Johnny2Bad on March 17, 2015 - 10:44pm

I've known a few Musicians with critical ears, but for the most part "musicians have the worst stereos". I believe it's because their brains process music (not sound) differently. They "play" the notes in their heads rather than listen to the music alone. They appreciate, but don't actually need, a good playback system.

rtaylor76's picture
Submitted by rtaylor76 on March 19, 2015 - 10:02am

I think non-audiophiles expectations of expensive gear is overrated. Meaning they have very high expectations given the price. They think the price is indicative of the quality (which is one of the reasons Bose is so popular), and it is a linear scale.

Non-audiophiles just don't know how to listen in the right way. They still think in terms of EQ (bass-mid-high) and think of sound as 2 dimensional. In a hi-fi shop is also not their own equipment to compare it to. It is also a different acoustical space. I worked at a consumer electronic stores in the early 2000's (best we had was low mid-fi gear) and we had speaker tower returns and subwoofer returns many times because it did not sound like it did in the store. They were unimpressed when they got home, even-though they A-B'd every speaker we had to death. Not everyone came back, but one or two a year at least. Not sure if it is source material, different acoustical space, or just buyers remorse.

But I am not surprised that many non-audiophiles were not impressed at first. I think many are curious, like "what will X-amount of money buy me?" They just think it will make their terrible sounding recordings will all of the sudden knock their socks off. But for those that know, there is is a certain subtlety to certain things, like phono cartridges or different DACs or even different amps. It takes a trained ear to catch those minor differences. Speakers there is more of a differences because it is entirely different physics and components involved. It took many years for me to train my wife, but she now can tell the difference in good and bad audio. However, when we are in a hotel room and are forced to listen to the tiny flat screen speakers, it bothers me a whole heck of a lot more than it bothers her.

lo fi's picture
Submitted by lo fi on March 20, 2015 - 5:42pm

Spoken like an audiophile.

Joe Whip's picture
Submitted by Joe Whip on March 21, 2015 - 7:04am

and I consider myself an audiophile. I listen to some really expensive equipment and am unimpressed. Some I have heard IMHO sounds terrible. When I express this opinion, I am told by others including a reviewer that I don't know what to listen for or I don't know how to listen. I attend numerous and I mean numerous live concert events a year in larger concert halls and small jazz clubs, here and overseas, amplified and acoustic. A good number of these individuals rarely attend concerts but tell me I don't know how to listen. It makes me laugh and also pisses me off too. Everyone knows how to listen. All it takes is a little concentration. Audiophiles should never tell someone not in this hobby that they don't know how to listen. It is so condescending.

hapinoregon's picture
Submitted by hapinoregon on March 21, 2015 - 11:45am

I learned a long time ago that I'd never reproduce in whatever room my stereo gear was the sound of center seat, first row, first balcony of almost any great hall in the world.

Once I made my peace, I began to look for a sound I could live with comfortably and enjoyably. Now, I go downstairs, decide what I want to hear, put it on, sit back and listen, at peace and without angst.

My system may be $3k. I don't want to know what my library is...

Joe Whip's picture
Submitted by Joe Whip on March 21, 2015 - 12:43pm

I agree completely hapinoregon. That is how I have done it. Spent a tad more because I bought some stuff used but not all that much more. I just spend money now adding to the music collection and have more many years.

GabrielS's picture
Submitted by GabrielS on March 25, 2015 - 4:49pm

Way back in my early days in high school I was bitten by the sound bug. Always out to improve the reproduction but limited by budget, I found a vast resource of parts and supplies at local thrift shops. My intent was building speakers and used lots of plywood my father brought home from work (not his intended use) in making various sized enclosures with various types of components. Not very well versed in electronics nor understanding amplifier loads, I recall a small cassette recorder that I kept adding speakers to the headphone output jack until one day there was no sound. I wired the speakers in parallel to the point there was no load on the tiny amp and it gave up the ghost.
I remember visiting Dad in my adult life still seeing the remnants of my speaker building foray scattered all over the garage. Lots of circles and odd sized rectangles of wood. There was even a small gutted speaker being utilized as a support for one of the car axles he was working on. Being made of 3/4" and 1" plywood, I guess I built them fairly strong.
Still an avid audiophile with many speakers with that insatiable quest of better sound and collecting components I could only dream of having as a kid. My wife is a good sport and somewhat tolerant about it, trying to understand my need for so many systems that encroach into our living space.

stephen jones's picture
Submitted by stephen jones on March 25, 2015 - 10:42pm

I like to listen to both sound and the music.
(Neil Young is right, I reckon : the recording should transmit what the Musicians intended)

My early system was the Luxman PD181(?)turntable with Grace g707 and the Supex Super 900. The amp was a Crown ; the speakers were made by Spendor.
I do not remember the model designation - the 70's were ages ago - yet now, with digital music, I can take all this with me, practically, in my pocket.
There's nothing tinny about getting the best sound possible with the best music available. The Musicians, I'd think, would expect it of us.

bdaddy62's picture
Submitted by bdaddy62 on March 26, 2015 - 1:30pm

Its kind of sad we chase the wind in so many silly ways...audio equipment and cars for example. They are both designed to take us somewhere.. simple right ? Our heads fuelled by advertising make that the game it is. I don't care what kind of car I have as long as it gets me where I need to go. I don't care how much an audio system costs if it gets me some music......really this subject is sooo boring and redundant it needs to go away.......we're being distracted.

X