Art Dudley Listening

The Pepsi Challenge (or taste test) is still the gold standard!

Talk about wrong on an epic scale.

The entire argument is based on false equivalences and just plain falsehoods.

There is absolutely nothing inherent in blind listening that demands listening only in snippets, make snap judgements, determine "which seems more real," or anything of the sort.

Why do such lies continue to be perpetuated? Is it out of ignorance or is it done intentionally?

I don't think blind testing in the comfortable surroundings of the home and over an extended period of time is a bad idea. Maybe the above poster would volunteer his time toward the effort? Art could put you up in the spare bedroom for a couple of months.

Actually the late Tom Nousaine set up a number of people with ABX systems in their own homes so they could test whenever they wanted to, for as long as they wanted to and using their own system.

" Are we disappointed when an otherwise good electronics manufacturer lowers its manufacturing costs by switching from hand-wired circuits to PCB construction, because the company was persuaded that "that doesn't make any difference"?"

Good Lord, please spare us this baloney. Hand wired circuits? Other than pulling out a bunch of myths that have no backing in fact, please explain what advantage there is to hand wiring over pcb's. Never mind that no two hand wired circuits are ever the same. Or that your wiring paths are often longer and your chances for joint failure are far greater. Same goes for crosstalk, hum induction, ground loops, etc.
Of course hand wiring isn't even an option for most transistor circuits, so I guess we're mired in the wonderful world of 1930's amplifier technology too.

"Are we disappointed when a manufacturer of classic loudspeakers begins making cabinets out of MDF instead of plywood, because an engineer convinced the company that "that doesn't make any difference"?"
How about the entirely plausible explanation that MDF is actually BETTER than plywood from a rigidity and self damping perspective, and you can make better cabinets by using it?

Other than manufacturers testing variations of their own products, no one can, has ever done, or will ever do DBT's of audio equipment that shows any significant results, "scientifically", statistically, or otherwise that will be meaningful to the average consumer, let alone to "hardcore" audiophiles.

John Atkinson knows so... So does anyone who understands research methodology.

Articulate and entertaining as always.

I concur that dbt is meaningless as applied to home audio gear. The audio hobby is, by definition, an aural experience; there can be no greater arbiter than my own ears in choosing gear for myself.

If others wish to be guided by numbers on paper, may they be happy in their way as I am in mine.

Dear Mr. JA & AD ,
It seems a debate about this matter is rather pointless , the entire World went Digital .
I speak as someone that owned more Koetsu Carts than most people own Interconnects , I was a Importer of British stuff in the 1970s , Esoteric Audio in the 1980s ( retail ) and manufacture of OEM stuff for High-End as HP would call our Industry .
The Audio Hobbyists that own thousands of Vinyl ( Todd the Vinyl Junkie with 12,000 being a typical example ) cling and support this old storage medium , a few newbies are trying out the Garage Sale $1.00 Vinyls on Roy Hall's stuff , maybe buying a Schiit head Amp thing for $120.00 , too bad ? , maybe but there it is ! , you can blind test that and get the consistent result of seeing people walking down any street anywhere on the Planet with white wires hanging from their ears , it's obvious gentlemen : the marketplace listened and decided , perhaps blindly but overwhelmingly .
Vinyl still has a nucleus , East Coast , NY , probably Brooklyn or thereabouts . There , a person can live vinyl and feel at home , somewhat at home , at least able to find kindred spirits . Vinyl is becoming a Cult now , probably little more that a historical artifact for the balance of the Population ( 99.9999% or greater ) . The Glue that hold it together is that people "Save" their parents Vinyl and 78 Record Collections , for old time sakes and old family memories sake , something like the Photo Albums from yesteryear .
The whole game has changed , something I'm certain you are trying to cope with . HP just passed , his memory is being discussed , his position in the Industry will be filled , vacuums are always filled , most likely by a digital person , I'm reasoning , someone that understands Digital to Analog Converters perhaps your Tyll in the Frozen North , another bright penny is that Darko lad from Down Under , he writes like a Good Brit , very clever with words is he , ( might be a nice addition to your staff ) , does a nice job of kissing up to manufacturers ( advertisers ) .
Well , Gentlemen , Grado make Headphones now , haven't heard of a Grado phono cart mentioned in the last few years ( by anyone ) , kinda says it all . I know people that are going gaga over their 1000 , trying to work out a good amp matching . The Audio Hobby is Alive and exciting but not in Vinyl or Analog segments , sorry , it's over , dead and nearly forgotten .
You may still be right about Vinyl's excellence but you'll be dead right !

Tony in Michigan

I'm not one of those awful objectivists, but even I can see that this is a lazy opinion piece which resorts to diatribe and unconvincing analogies to prosecute a flawed argument.

Debate is great, but the article struck me as angry and defensive. Mostly, I thought there was a whiff of hypocrisy in this statement:

"While audio perfectionists tend not to care what's bought and used by people on the other side of the fence, advocates of blind testing are obviously driven apoplectic by the very notion that someone else is enjoying domestic playback in a manner that differs from their own."

Not really.

I've seen clever "set-ups" lead people , a blind test can be trickery , those Magic Show people in Vegas fool people all the time .
However , I used to do blind tests in my Audio Salon , comparing all my small monitors . I used Electrocompaniet , VPI , Koetsu and MIT 750 wire . We blind tested : Pro-Ac Tablettes , LS3/5a , Celestian SL 600s ( the Alum version ) , Quad ESL 63s , the little Kindels , and other small monitor anyone brought in . Consistent result was the Pro-Ac Tablette , every time . We even had customers perform the test for other costumers , same result . Tablettes were outstanding back then , they may still be , wouldn't know , I gone to Active Genelecs , would be a Meridian M2 Active owner it they still made em .
I guess that Blind testing can be a starting point , time reveals all , other considerations come into play and anyone can be fooled or tricked , but Blind Testing does help undecideds with little patience or self-assuredness make an informed ( kinda informed ) quick decision .
It is a safe position to worry about the results of a Blind Test , a person that's been burned will tend to deny the usefulness of it but we all are trying to eliminate our emotions from a scientific decision , Blind can help with that .

Tony in Michigan

Single blind testing can be fun and informative, but the emphasis by some on double blind testing is misguided and misplaced!

"To judge the authenticity, the effectiveness, the legitimacy of any work of art requires three things: a participant who is familiar with the work(s) at hand; a test setting that's appropriate to the appreciation of art; and a generous amount of time for the participant to arrive, without stress, at his or her conclusions."

This is bullshit. 1) Equipment is not Art. Art can not be assessed on the "usefulness" scale. 2) Art appreciation, Art 'legitimacy' has nothing to do with 'it' not being a forgery. Art appreciation requires nothing of the subject, Assessment of the real Art vs Fake Art is a "guys in white lab coats" thing, This is 'Art' on sterile cellular level. Molecules being compared to other molecules.

Art- seriously, no pun intended.

" 'Coke's problem is that the guys in white lab coats took over.' Took over and got it wrong, I would add. Wrong on an epic scale.'

I wont argue with you there - it's not like back in the day :)

Both of these examples go straight to the false equivalences that make up the "argument."

With audio testing, at least of the sort that's being discussed here, the task of the listener isn't to identify the fake from the original. It's simply to determine if they can reliably detect a difference between two presentations.

But this false equivalency is further reinforced by saying "Notably, those are the same challenges confronted by the listener who wishes to distinguish the sound of one piece of playback gear from that of another: He, too, must examine two samples of the same work of art in an effort to determine which seems more real."

That's complete nonsense. The listener isn't tasked with determining that which "seems more real." Just whether or not there is a difference. Even a layperson could probably look at an original and a forgery and accurately tell you if there's a difference between the two. Being able to tell you which is the forgery is an entirely different matter.

And the Pepsi challenge example is even more egregious. Art says "Time and again, the use of blind testing in an attempt to quantify human perception has been discredited, most recently in the best-selling book Blink, by journalist Malcolm Gladwell."

This is complete bullshit, and it's sad to see Gladwell's work used to make such a bullshit argument.

The Pepsi Challenge wasn't a difference test, it was a preference test. The "sip test" didn't prevent anyone from determining if there was a difference between Pepsi and Coke. That's not even what was being tested for crying out loud! The Great Audio Debate has never been about preference. There's simply no arguing peoples' personal preferences. What, are you going to argue with someone that their favorite color isn't blue?

Gladwell's argument was that the "sip test" didn't necessarily reflect peoples' preferences compared to drinking a whole can or bottle, which is how most people consume soft drinks. It is NOT any sort of discrediting of blind testing as a whole as Art claims it is.

I simply don't see how any rational, thinking person could, in good faith, put forward the arguments that Art did.

This is a sad hypocritical article, and Stereophile should be ashamed to have published it. More engineering shaming as if any of the equipment you so cherish would exist without them.

Art/John, if blind testing is so faulty because it requires snap judgments, why does every subjective review start of with "I heard a dramatic difference immediately" or something similar? Good enough for subjectivism but not actual science?

As always, these arguments are the same made by religious zealots. Ugh.

Bla bla bla is what I hear I my head when reading most of the articles in Stereophile. Keep up the discouraging writing.

Quote:

JA - Do you see any signs of future vitality in high-end audio?

JG Holt - Vitality? Don't make me laugh. Audio as a hobby is dying, largely by its own hand. As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me, because I am associated by so many people with the mess my disciples made of spreading my gospel. For the record: I never, ever claimed that measurements don't matter. What I said (and very often, at that) was, they don't always tell the whole story. Not quite the same thing.

Reliable testers get around mistakes and bias by repeating tests at different times, or changing some of the parameters of the tests. Reliable testers are good at self-policing, and good at observation. Observation is important because you can never assume that a testing script is good enough - those have to be analyzed or tested like anything else. Reliable testers are the best skeptics, because they know that differences lurk everywhere and can trip up the best-laid plans. Reliable testers know that their reputations can take a hit if they take shortcuts or cover up incomplete work.

!. The listener. How many people, self acclaimed audiophiles, etc. are really trained in how to listen? Harmon has an application and testing procedures where they certify their listeners. They first have to pass all of the levels of each test on their application, THEN the person has to go down to Harmon Labs and get retested to ensure that they didn't guess or cheat in any way, and they also have their hearing checked. I have downloaded the app and have only taken a few of the tests and eventually got to Level 6, but I still have a ways to go and I simply don't always have a good environment since I live in an area where there is just too much outside noise and that leaks into my home environment so the noise floor is simply too high during the day. It's also a very time consuming application because it's not that easy once you get past about level 4 or 5 with these tests. I wish ALL magazine/on-line magazine reviewers of gear would become certified through the Harmon program because it would help give these reviews a little more credibility.

2. If the listener is NOT used to the room, the system and the setups being AB'd prior to being blind folded, it's going to be more difficult. Example. Familiarity with a system, room, content is important to pick out subtlties. Heck, when I get a new pair of speakers, etc., it takes me a while to figure out where the changes are because I have to go through a large catalog of music and literally spend months listening to whatever change I made to figure out how well the change is. It takes time for one's ears/hearing to get accustomed to a system. Sure, big changes in a system are pretty noticeable, but the differences in cables, amps, etc. are not HUGE changes all of the time, they are subtle, but still many times noticeable and it's just getting familiar enough to know where those differences are.

3. Adding switch boxes and additional cables. I'm sorry, but if there is added equipment to conduct an ABX test that's not normally used, then it shouldn't be used. Unfortunately, you can't compare much audio gear against another in a quick ABX test without adding switch boxes and cables. these devices have measurable amounts of resistance, capacitance, inductance or any combination and they can invariably make whatever you are trying to compare sound more like one another. Cables especially since adding cables to another cable is just not done in real life.

4. Listening SPL level, what passages to listen to and having control over that must be in the hands of the listener. If someone puts the SPL too high, it may cause short term hearing damage if the average SPL is above 85dB, or you may or may not be able to tell differences between equipment at only one level, sometimes you have to adjust the volume levels to hear what differences there are at lower SPL and then higher SPL, plus the listener must have control over what the volume is set at, what content they are listening to and be able to replay certain excerpts that they are really familiar with. Most ABX tests don't always allow the listener to have access to the volume level and control what the content is, etc.

5. Is the room soundproofed and with good room acoustics? Most people don't have this and that can make a big impact on how well we hear what's going on in a system. Are listening tests being done in perfect environments? Not always, especially at a trade show with hotel rooms in an environment where people are constantly going in/out of the room, talking, and making other noises.

6. How about those that listen to a product in a showroom at a dealership and then they take the product home and it doesn't sound anywhere like it did in the showroom? I can't stress how important room acoustics are and how that impacts what we hear or perceive in any listening of audio equipment.

What I see is a lot of relatively ignorant people that simply don't have much money or a high degree of experience with high end audio gear simply making up excuses to put down mfg, high end magazines, and "audiophiles" and they use the ABX test as some sort of valid reason. It's just as invalid as anything else. Yeah, I'm not suggesting that every audio mfg is telling the truth, etc. etc. Sure, in any industry there are those companies that market their products with a lot of suspicious claims, etc. And there is no real specific measurements that all mfg have to adhere to, and that measurements alone are not going to tell us how well something sounds all of the time. Plus you simply have to deal with people's subjectivity and the fact that everyone has a different skill set in terms of how to listen to audio gear.