Some folks want closure on every question - some don't think it matters. Some are control freaks - some aren't. Life on Earth. Often unpleasant, but, always worse without Music Music isn't for everyone ... sigh.

It's the focal spot for two completely different approaches to our hobby.

The reason that it becomes so hot is probably that both sides argue as if they had the "truth" but unfortunatley for every one the "truth", if there is anything like that in this matter, is problably missed by both sides due to the simplifications done in the debate.

If the debate could be lowered a couple of keys into a discussion it might be possible to find common ground...

Everyone no doubt can add a different twist to the question of blind testing. I personally like it very much, as a fan of the scientific method. What I dislike is the use of blind tests that failed to measure a difference as proof that the difference does not exist. It seems inappropriate to me to condemn as wrong opinions that are not supported by double blind tests.

On the other hand, I think many people are deeply disappointed by spending money on equipment that receives great reviews, only to fail to hear an improvement in their system. What is missing in many cases is basic knowledge that first speaker placement is the biggest improvement per dolllar that can be made in many cases. Next may be room acoustical treatment. Only if a genuinely high quality sweet spot is created are more subtle measures such as amplifier changes liklely to become really important.

Finally, there is a need many feel to defend science against mysticism in a society in which Astrology is often confused with Astronomy. I think that many see blind testing as the shining light of science into a realm of myth. As a scientist I sympathize, but I have to disagree that blind testing of audio equipment has proven to be as useful as we all hoped it would be.

I think it is the same reason that some people have a need to convert others to their religion, their profression, etc. They need validation. They need to feel they are right. I say let them have their debate. I would rather have fun. At least I'll die with a smile on my face.

People just want to be right and have a definitive answer. Human nature or immature squabling or whatever you want to call it...it just people.

Ed

>As the HE2005 Debate thread reveals, blind testing stirs strong passions, pro and con. Why _is_ this?

Because it questions the very reason for the existance of the high end audio industry - do you really have to pay the price, both practical and financial, of high end equipment to have the best possible sound quality.

>Life on Earth. Often unpleasant, but, always worse without Music Music isn't for everyone ... sigh.

How does this relate to DBTs?

>It's the focal spot for two completely different approaches to our hobby.

What are those two approaches?

>What I dislike is the use of blind tests that failed to measure a difference as proof that the difference does not exist.

What would you consider to be adequate support for the idea that there is no audible difference between two pieces of equipment?

>I think it is the same reason that some people have a need to convert others to their religion, their profression, etc. They need validation. They need to feel they are right.

Who is this "they"? High End true believers or the scientific skeptics?

>I say let them have their debate. I would rather have fun. At least I'll die with a smile on my face.

Who are you suggesting isn't having fun?

Blind testing is supposed to limit bias; but in my experience with it, I could not trust the results. In the one I participated in I was easily able to tell the difference between two power cords outside of the test. During the test I could not. After the test I could. So what does that tell us? Either the blind test reveals the true nature of the sound differences, or lack thereof, or the testing procedure is fundamentally flawed in the way it is implemented in audio evaluations. The test I would like to see is this. Two identical rooms, two identical systems, with only one change, one is wired with boutique cables the other wired with stock cables. Participants go into room 1 and listen to a complete CD; none of this ABX business where you hear a little bit of a track with cables A, then switch to cables B, and then on to X, which you have to match to either A or B. To me that is no way to judge the sonic characteristics/differences between like components. I digress though, after the participants have listened to the CD in Room 1 they go to Room 2, they listen to the complete CD there, and then write down their impressions of both rooms. Same or different? If different how so? Now that is a blind test I could sink my teeth into and actually trust the results

>>What I dislike is the use of blind tests that failed to measure a difference as proof that the difference does not exist.

>What would you consider to be adequate support for the idea that there is no audible difference between two pieces of equipment?

Data that would be of interest to me personally would consist of measured 24-bit comparisons (subtracting one data set from another) of a comprehensive set of systems using the best available recording technology. If the difference data sets were always equivalent to the recorded noise, that would be good support for no significant differences. These comparisons would be between complete systems, from source material to speakers. Tests would include temporal dispersion, frequency response, THD, and IMD using a large number of simultaneous tones (not just two). The definitive tests would be made with music sources. When I listen to large orchestral ensemble violin passages, the varying amount of "congestion" (for lack of a better word) that I always hear is apparently a function of the equipment used. I would be interested to see comparative measurements. Given that the data sets will probably measurably differ from noise, it will be far more difficult to decide on the threshold of audibility.

I think that most people have experiences and beliefs wired hard enough that they have trouble empathizing with The Other Side. When the one side is naive, unscientific easily fooled, while the other side is deaf, poor, inexperienced and overly idealistic, there isn't a whole lot of room for empathy.

>>>What I dislike is the use of blind tests that failed to measure a difference as proof that the difference does not exist.

>>What would you consider to be adequate support for the idea that there is no audible difference between two pieces of equipment?

>Data that would be of interest to me personally would consist of measured 24-bit comparisons (subtracting one data set from another) of a comprehensive set of systems using the best available recording technology. If the difference data sets were always equivalent to the recorded noise, that would be good support for no significant differences.

These are just measurements. Unless you accept some pre-defined mapping between measurements and audibility, then they are meaningless.

I was hoping for a meaningful answer, not just a naive suggestion that we rely a bunch of abstract numbers.

>Blind testing is supposed to limit bias;

So, do you have any comments about that? You quickly change the topic away from this issue!

> but in my experience with it, I could not trust the results.

What are the grounds for that, aside from your disorganized anecdotes?

> In the one I participated in I was easily able to tell the difference between two power cords outside of the test.

Yup, its pretty simple tell a difference in a sighted evaluation. Perhaps one is green, the other is blue. They're different! Oop, this was supposed be about sound quality not cosmetics, right?

>During the test I could not.

As soon as the evaluation was just about sound quality, the perceived differences, which were based on sight and foreknowlege sort of disappeared. How about that?

Any reasonable person would look at this situation and say: "Well, they looked different but they did not sound different". Apparently, true-believer audiophiles lack the mental abilities of any reasonable person.

> After the test I could.

One was still green and the other was still blue.

>So what does that tell us?

Any reasonable person would look at this situation and say: "Well, they looked different but they did not sound different". Apparently, true-believer audiophiles lack the mental abilities of any reasonable person.

>Either the blind test reveals the true nature of the sound differences, or lack thereof, or the testing procedure is fundamentally flawed in the way it is implemented in audio evaluations.

The obvious conclusion is that the power cords looked different but they did not sound different.

>The test I would like to see is this. Two identical rooms, two identical systems, with only one change, one is wired with boutique cables the other wired with stock cables. Participants go into room 1 and listen to a complete CD;

Tests like this have been done. No joy for the guys selling magic cables.

none of this ABX business where you hear a little bit of a track with cables A, then switch to cables B, and then on to X, which you have to match to either A or B.

There's nothing other than a bunch of misled audiophiles that say that ABX tests must only involve short term listening experiences.

>To me that is no way to judge the sonic characteristics/differences between like components.

It turns out that if you pursue long-term versus short-term listening, you find out that long-term listening is very insensitive comapared to listening for periods of time that give the most sensitive results.

> I digress though, after the participants have listened to the CD in Room 1 they go to Room 2, they listen to the complete CD there, and then write down their impressions of both rooms. Same or different? If different how so? Now that is a blind test I could sink my teeth into and actually trust the results.

There are many problems with the proposed test, including the fact that when people just write down their impressions, it is difficult or impossible to quantify their results without a large number of judgement calls by the people reading the impressions and trying to quantify the results. It's an inherently fuzzy test.

Secondly, an opportunity to estimate the reliability of the listeners has been missed because there is no formal comparison between what they say they heard versus what they actually heard. The fact that only fuzzy data was collected just exasperates the situation.

Things like this has been done, and the results matched

>I think that most people have experiences and beliefs wired hard enough that they have trouble empathizing with The Other Side.

It appears to me that the following statement shows that you want to be a walking poster child example of this problem.

>When the one side is naive, unscientific easily fooled,

That seems to describe a lot of high end audiophiles.

>while the other side is deaf,

How do you know that?

> poor,

How do you know that?

>inexperienced

How do you know that?

>and overly idealistic,

How do you know that?

>there isn't a whole lot of room for empathy.

..at least none demonstrated in this post.

I find that high end audiophiles are as a group pretty smart people, many who were caught up in a well-orchestrated false belief system. One key to that false belief system involves staying ignorant of audio knowledge that would change their belief system.

For example, the study of psychoacoustics naturally leads to the conclusion that many measurable differences between audio equipment shouldn't be audible.

The study of electronic measurements and technology as applied to cables leads to the conclusion that it is unlikely or impossible that many vendor claims about cables could be true and or relevant.

Given the above demonstration of ignorance and prejudice, the idea that many audiophiles are ignorant and prejudiced seems to be gaining support.

The study of experimental design leads to the conclusion that the evaluations that most high end audio writers are grotesquely flawed and insensitive.

JA:

I believe that you get two different questions mixed up.

Quick swithing between test objects is one issue and blind testing is another one.

Blind testing does not require quick switching between the test objects. Blind testing is just a way to disconnect the eyes and mind from the knowledge of which object is used for the moment.

I read your opinions in the As we se it section in the July issue and have just one question:

You more or less state that if you have had the Quad and the M&A tested for a longer period you would have noticed the differences. But what if you would have been blind testing during that extended time, wouldn't you have noticed the differences then? And the differences might have been statistically secured... Everyone happy?

Does the term "High-End Audio" automatically imply megabuck pricing?
Is mega-pricing the sole determinant of what constitutes "High-End Audio"?

Arny, the wording

Quote:
the high end audio industry

is another term that is both emotive and linked to the automatic implication that "high-end" equals "high-price"..

I'm not sure I buy this "hidden definition"...

Common usage has an in-built aspect of obsolescence.

Take the term "High Fidelity" (contracted to "Hi Fi") in the context of what it was intended to mean at the time it was coined - now, take the term in the context of how advertising for plastic mini-systems has "bastardised" it. Every piece of cruddy, plastic, Star Wars console look-alike is called a "Hi Fi" - irrespective of any ability to faithfully reproduce anything...

Result: scratch the term "Hi Fi" from the lexicon of serious audio - its obsolete...

Next, we have the word "audiophile" (when used as an adjective by - hold your breath - advertisers.... Now that the term "Hi Fi" has been devalued to the point of meaninglessness, we see the same crappy mini-systems described as "audiophile-quality mini-systems" when, once again, they are still incapable of any true high-quality audio reproduction...

Result: scratch the word "audiophile" from the same lexicon - its also obsolete...

Currently, the term "High-End" has had its scope of definition enlarged to imply any design in which sound quality is the primary consideration - irrespective of price (source: Robert Harley)

So, let us summarise: the term "High-End" is in the process of being debased to encompass any design with a focus on sound quality - this encompasses virtually all of the serious manufacturers of audio equipment - even some of those that produce the "flashy crap" referred to above.

This renders it "null" in the context used in your post.

Maybe its time to coin some new word for you to use?

Quote:
>Life on Earth. Often unpleasant, but, always worse without Music Music isn't for everyone ... sigh.

How does this relate to DBTs?

DBTs are an approach to selecting components. A few of us do not think the selection of components is the only interesting part of this audio hobby. It is possible I have all the wrong pieces. I don't care, the Music is still the fun part. (I may have the right key component - attitude)

The Hot Button is that one can be proven "wrong" (this is the rub) is to prove that you can HEAR a difference between components or that others might think the amp/CD player/speaker "I" chose is not the better of the two. I think that to hear differences and to chose which one is better is two different challenges.

I would consider it a challenge to be able to succeed at DBT just to validate (if that is possible) that I can hear small differences between amps or CD players at a very high percentage of chances. I do not think that at my age of 58 and hearing loss that I could participate in cable DBT.

I do agree that how the test is conducted is critical. I do think it would be "fun?" at HE 2006 for trial to be done with visitors and see how it all plays out. Some manufacturers might be willing to help promote and particpate, others may prove not to be exposed to this level of scrutinity.

We must admit that it still comes down to human error, preference, and a high level of experience of critical listening. I am guility of listening, but I could not say that I am a trained critical listner. Others who have listened to the worlds best equipment for a long period of time would be a better choice. Names that come to mind are Al Schmitt, Bob Ludwig, Roger Nichols, Tom Jung, Mr. Meitner and... members of the Stereophile family, and of course others. I do think that recording engineers and mastering engineers should have better success in DBT. It is their life's work to hear such differences and make choices that determine what we finally hear.

To expect absolute proof for or against DBT will probably not happen, but be a good experience for all. It will allow the debate for or against DBT to continue on. Trying to get people to agree about certain things in this hobby is near impossible.

Quote:
>As the HE2005 Debate thread reveals, blind testing stirs strong passions, pro and con. Why _is_ this?

Because it questions the very reason for the existance of the high end audio industry - do you really have to pay the price, both practical and financial, of high end equipment to have the best possible sound quality.

How on earth does the price of equipment have anything to do with dbts? But yes, you do have to pay the price to get the best sound available at any given price point. That should be painfully self-evident.

Quote:
Some folks want closure on every question - some don't think it matters. Some are control freaks - some aren't.

As a skeptic, I don't want closure. I must be prepare to change my views everytime new evidences are found.

Quote:
Life on Earth. Often unpleasant, but, always worse without Music Music isn't for everyone ... sigh.

Condescending and irrelevant.

This I do not buy into. What certain parameters? As a broadcast engineer there is much more to it than that. I would agree that money(cost) and high end do not guarantee sonic superiority, but DBT without knowing the manufacturer and then decide if YOU can hear any difference between two pieces of gear will tell you your sonic accumen and,or preferences.

I believe it has more to do with the listener be found a superior listener, or NOT, not whether the gear sounds different and then determined to sound better. What did Marley say to E. Scrooge: Why do you not trust your senses? If bad mustard or a slightly under-done potato does you in well...

These differences could become interesting ie. a simple test of taking two Jolida 1701's or two 1501 integrate amps with tube preamp stages and mosfet outputs. Put different preamp tubes in them, the stock ones are Chinese 12AX7's. Put Svetlana's, EH's, Sovtek's in the other, anything you want. Then later get real crazy and put NOS Telefunkens and Mullards 12AU7's in one and continue to see if you can pick with a high percentage. It is easy to adjust for the tube gain differentials. It might also give you a chance to see if you can judge the linearity of the vol pot at differing gain positions.

Instead of considering all of this to be a personal affront to your sensibilities, test your own abilities in private, then no one will know but you. Anything that would help train me to become a better listner would be welcome to me.

Whether DBT proves anything about your ability is one debate, but I do believe that practicing and learning to be able to discern sonic differences without a visual prejedice is important if you want someone else to trust your opinions about equipment as to merit their audio ownership.

Quote:
Any reasonable person would look at this situation and say: "Well, they looked different but they did not sound different". Apparently, true-believer audiophiles lack the mental abilities of any reasonable person.

Arny, if you want to inform people, questionning their mental abilities is not the way to go. It's rude and counterproductive. Audiophiles are not "true-believers". Au contraire, they are "pseudo-sceptics". They don't accept any evidences. If they believe in something, they believe "there is maybe something we don't know." They believe in possibilities, even if we don't have any good reason to believe in those possibilities. It's very hard to demonstrate that they don't have "good reasons" to doubt. Good luck with that !

Sorry guys, forgot to log in before I made the above post. But yes, it was from me.

John Atkinson
Editor, Stereophile

Quote:
It comes down to a debate on whether to accept
authority from external sources or to exercise the freedom
to make one's own choices and accept that those choices may
not always be right.

I'm not sure I'm following you. A DBTest is a subjective experience. So the choice is between two subjective experiences. In either case, we must accept the fact that we can't always trust our senses.

Quote:

Quote:
It comes down to a debate on whether to accept
authority from external sources or to exercise the freedom
to make one's own choices and accept that those choices may
not always be right.

I'm not sure I'm following you. A DBTest is a subjective experience. So the choice is between two subjective experiences. In either case, we must accept the fact that we can't always trust our senses.

But you can't necessarily substitute the trust of other
people in their senses either. This was the point of the
parable I told at the HE 2005 debate. (See my July AWSI at
http://stereophile.com/aswesee it/705awsi.) The formal blind
test had informed me that the Quad amplifier that I wanted
to buy was a good as any other. Yet over time it became
glaringly apparent that I had been wrong to put my trust in
the results of that test.

A more recent example: on the radio last week a reporter
said that MP3s at 128kbps are indistinguishable from the
original CDs for more than 99% of listeners on 99% of
music recordings. Should I have accepted that test result
instead of relying on my own experience, having done
comparisons of MP3s at all data rates and finding that
128kbps files invariably sound like rubbish?

John Atkinson
Editor, Stereophile

John, I agree. I have friends who cannot tell the sonic differences between my cheap Sony 755 DVD/SACD player and my Jolida JD100 tube cd player. I find this surprising as I then play the same discs through both, and even play the 2 channel SACD through the 755 and the CD version through the Jolida. It is clear to me that I enjoy the smoothness and the low end fullness the Jolida provides. I can get a glimpse of the smmothness that SACD offers, but I am convinced that you must enter $1K SACD player land before you can begin to enjoy what the format has to offer.

I am amazed at what some miss. This is the test for me that buying a great CD player is much more important than just owning SOME SACD player. I probably will one day buy a player equal to the improved format. I really bought the 755 for video in a spare system.

I have done the same thing with my Jolida and comparing it to $2K Cairn Fog with 24/192 upsampling and the $2500 Audio Analogue 24/96 upsampler. This is where it becomes tougher. My wife Diane, who is no audiophile, could describe the characteristics of each player very well to small details. It was harder for her to state which one she preferred as the variations in software made her like one over the other.

Then for fun I play a Mini-Disc copy of a cd and let them try and hear the loss of definition and 3-D that the MD loses in its compression algorithm from the Jolida. What is obvious to me is lost on others. I do not think less of them as people, but wonder why is it that we hear so differently or cannot discern or care about the differences? This is certainly not as difficult as DBT.

It may be that they are listening TO the music, but not INTO the music as deeply as audiophiles tend to do. Maybe they are better off. The debate rages on.

Quote:
The formal blind test had informed me that the Quad amplifier that I wanted to buy was a good as any other.

No, it actually did not.

If you had been able to hear a difference during the blind test then you would have known that there was a real difference.

But the opposite is not true. The first law of science states that you cannot prove the absence of anything. The result of the blindtest was negative and did not prove anything at all, certainly not that the amplifiers sounds the same.

The only possible conclusion is that you were not able to hear the difference right then, in that setup, at that day, with that music etc...

edit: I hope the tone is not too harsh.

>I was hoping for a meaningful answer, not just a naive suggestion that we rely a bunch of abstract numbers.

The numbers I suggested quantify the difference between systems, which is not merely abstract. It is related to the mean squared error method that I used to test the data compression algorithms for the Imager for Mars Pathfinder, for example. I would think that there would be some interest in quantifying the difference you are attempting to detect with DBT. I also pointed out that the audibility question remains: I did not suggest we "rely" on this data alone. I think it would be fun to play back the difference signal at the original volume just to hear what level of differences we are really talking about. That is not a scientific enterprise, but it might build insight.

Quote:
>It's the focal spot for two completely different approaches to our hobby.

What are those two approaches?

I did not write that quote, but I'll answer anyway, since I think I have pretty good idea what HiFiforum.nu means by that.

The approaches are (I think):

1. To get what best experiences from your music collection and thus wanting a system which sounds the way you want it to, giving you the most enjoyment as a result. By this approach, what sounds good, is good! I would describe this as the most widespread HiFi hobby.

2. To be able to listen to the recordings without any unnecessarily added color or substance. Wanting, simply, to be able to hear what the musician/composer/moviemaker wanted us to hear. Scientific studies seems to me the clearest way to reach this goal - taking into account every possible trap which stands in the way of being able to _re_produce any sound with your system.

I don't really know why blind tests are so hot. There are quite may amps and CD-players that are detected in single-blind tests by the Swedish Audio-Technical Society. We presented data on a DBT on Audioreview a few years ago, regarding two CD-players (Denon vs. H/K). It was brought up on rec.audio.high-end, a few years later.

Most amps and CD-players are fine in home environment and during normal listening levels though.

>DBTs are an approach to selecting components.

Among other things.

>A few of us do not think the selection of components is the only interesting part of this audio hobby.

I agree that using equipment is generally a lot more fun than choosing it.

>It is possible I have all the wrong pieces. I don't care, the Music is still the fun part. (I may have the right key component - attitude)

I find that recording live music and listening to it is a lot more fun than messing with the hardware.

> The formal blind
test had informed me that the Quad amplifier that I wanted
to buy was a good as any other. Yet over time it became
glaringly apparent that I had been wrong to put my trust in
the results of that test.

The obvious fallacy here is Atkinson's constant implicit assertion that all DBTs are equally trustworthy. OK, so Atkinson did one tiny little DBT 20 years ago that mislead him. Is that one test 20 years ao the gold standard by which all DBTs, past present or future should be judged?

For example, the ABX development group quickly determined in the late 1970s that the choice of musical selection was exceedingly important when it came to revealing differences between components. This knowlege became well known within the audio subjective testing community by the middle 1980s.

We really know nothing about Atkinson's 1978 Quad amplifier tests. There's no doubt in my mind that were they to be done today, using the best knowlege we now have about DBTs, considerable light might be shed on the subject.

>We really know nothing about Atkinson's 1978 Quad amplifier tests. There's no doubt in my mind that were they to be done today, using the best knowlege we now have about DBTs, considerable light might be shed on the subject.

If the claim is that DBT is much more precise (and presumeably more sensitive in difference detection) now than it was even a few years ago, then perhaps the next HE2006 event should be (instead of a debate) a DBT challenge in which the best set of professional (or other) listeners that John Atkinson can assemble participates in the best DBT's that each of JA and AK can create (in an off-site dedicated listening room, not with show background noise) to see if a significant difference is detected between, for example, a $200 integrated amplifier, a NAD 370, and a Halcro dm10-dm68 set up.

Quote:
>We really know nothing about Atkinson's 1978 Quad amplifier tests. There's no doubt in my mind that were they to be done today, using the best knowlege we now have about DBTs, considerable light might be shed on the subject.

If the claim is that DBT is much more precise (and presumeably more sensitive in difference detection) now than it was even a few years ago, then perhaps the next HE2006 event should be (instead of a debate) a DBT challenge in which the best set of professional (or other) listeners that John Atkinson can assemble participates in the best DBT's that each of JA and AK can create (in an off-site dedicated listening room, not with show background noise) to see if a significant difference is detected between, for example, a $200 integrated amplifier, a NAD 370, and a Halcro dm10-dm68 set up.

The Halcro dm68 is fine although it has been detected audibly different using the normal input. The DC-coupled (or very-near DC) input is better. I would also suggest a wire bypass test to increase sensitivity and a loudspeaker system with high SPL/low distorsion down to 10 Hz.

>> The formal blind test had informed me that the Quad
>> amplifier that I wanted to buy was a good as any other.
>> Yet over time it became glaringly apparent that I had
>> been wrong to put my trust in the results of that test.
>
> The obvious fallacy here is Atkinson's constant implicit
> assertion that all DBTs are equally trustworthy. OK, so
> Atkinson did one tiny little DBT 20 years ago that mislead > him.

I refer you to the recording of our debate, Mr. Krueger, in which I clearly listed the very large number of blind tests in which I have particpated over the past 28 years. This recording is available at http://www.stereophile.com/news/050905debate. I mentioned the 1978 test as being representative, not because it was the only such test in which I have participated.

> We really know nothing about Atkinson's 1978 Quad
>amplifier tests.

They were reported in exhaustive detail in the November 1978 issue of Hi-Fi News. My point is that at that time, I had no reason to doubt the results of the tests. And as I reported in the July Stereophile, I told this anecdote at the debate to make two specific points. First, it demonstrates that my following the then-as-now "objectivist" mantra -- that audiophiles should buy the cheapest amplifier that offers the power and features they need -- had let me down. Second, it pits against one another two core beliefs of the believers in "scientific" testing: 1) that a blind test, merely by being blind, reveals the reality of audible amplifier differences; and 2) that sighted listening is dominated by nonaudio factors, the so-called "Placebo Effect."

To explain my quarter-century-old Damascene experience, you have to accept that either the blind test was flawed -- in which case all the reports that cited that 1978 test as "proving" the amplifiers sounded the same were wrong -- or that the nonaudio factors were irrelevant, in which case the criticisms of sighted listening based on that factor must be wrong.

John Atkinson
Editor, Stereophile

"Who are you suggesting isn't having fun? "

I am not suggesting that you, or anyone, is not having fun in this debate. After all, some people find fun in serial killing. I do not presume to suggest that debating a position which, in the end, will not change anyone's mind, and is thus futile (read: waste of time) is not fun. I am only suggesting that the debate over whether I, you, or the man on the moon can or cannot hear a difference in audio components is not fun for me. I would rather listen to music, which is fun for me. So, have fun without me.

"Who is this "they"? High End true believers or the scientific skeptics?"

Both. If you believe that differences do not exist, that the differences are in my mind, I am not sure what you have to gain by converting me to your position, other than the satisfaction that you have done so. In other words, saving me from myself, which is what religious fanatics have demonstrated a need to do. I promise you that you are not in my will, and every penny saved from not purchasing audio equipment will not be left to you. So, there being no pecuniary interest in your success to convert me and others, we are only left with either your need to be "right", and proving to me your rightness, or the need to save me from myself, or the need to save me from charlatans (read: the devil). Either way, the similarities between you and the religious zealouts are striking.

Likewise, I am not sure why those, like myself, who believe that differences do exist in audio components spend so much energy trying to convince you of those differences. For the same reasons outlined above, I trust. The only difference is that you and people of your ilk tend to crawl from under their rocks attacking the other side first, rather than the other way around. This issue has risen to religious proportion for both sides.

In the end, I think, the problem is that too many people are not comfortable with their own opinions, without the support of others. You do not feel there are differences in audio components? Fine. Good luck to you.

Quote:

Quote:
The formal blind test had informed me that the Quad amplifier that I wanted to buy was a good as any other.

No, it actually did not.

If you had been able to hear a difference during the blind test then you would have known that there was a real difference.

But the opposite is not true. The first law of science states that you cannot prove the absence of anything. The result of the blindtest was negative and did not prove anything at all, certainly not that the amplifiers sounds the same.

edit: I hope the tone is not too harsh.

You are correct, but as I also said at the debate, when you are young you think you know a lot more than you are older and wiser. And no, your tone is not too harsh (though some other posters in this thread _are_ skating close to the edge of civility, I feel).

John Atkinson
Editor, Stereophile

Quote:
To explain my quarter-century-old Damascene experience, you have to accept that either the blind test was flawed -- in which case all the reports that cited that 1978 test as "proving" the amplifiers sounded the same were wrong -- or that the nonaudio factors were irrelevant, in which case the criticisms of sighted listening based on that factor must be wrong.

Not to be too impertinent, but another possibility is that in 1978, you were not able to tell differences between amplifiers, period, and that over time nonaudio factors degraded your experience with your Quad amplifier once you were in a nonblinded situation.

But getting back to the original topic, "Why is Blind Testing Such a Hot Button?", I think many of the same issues that make discussing religion such a hot button topic apply here. I have been interested in audio equipment since my college days in the early 80's. I am an amateur musician, and have spent a lot of time listening to live music, so I like to think that I have a pretty good ear and know what instruments and live vocals are supposed to sound like. I've spent more on my system than any of my friends would ever consider spending on audio equipment. But many of the statements that I've read in reviews of audio equipment just don't relate to my listening experience.

When Michael Fremer goes on about the deeper levels of blackness of one super high end turntable setup versus another, that just doesn't exist for me when I've done the same comparison. It's not that I have tin ears, it's just that I've not heard a turntable that was any better than my first experience with a Rega Planar 3. Likewise, when some of my friends bring up the changes in their lives that their particular faith has brought them, that just doesn't relate to my experience with my faith. In neither case am I going to be convinced to change my core beliefs. The chances that I'm going to convert to, say, Judaism, are probably the same as the chances that I'll actually buy a turntable more expensive than a Rega Planar 3.

It's clear that John Atkinson's experience with blind testing does not relate to his overall experience with audio equipment, so it's unlikely that he will become a believer in the validity of blind testing. So it's not inconceivable to me that the subject of blind testing can elicit such strong responses from people on either side of the fence.

Subjective reviews, I think, require that the reader place a certain amount of faith in the reviewer. And getting people to that point does take an epiphany of sorts. The problem is, over time, enough ridiculous claims have been made in the high end audio field that it's hard to tell the wheat from the chaff.

> There are quite may amps and CD-players that are detected in single-blind tests by the Swedish Audio-Technical Society.

Isn't a single blind test just a double blind test with some critical controls missing?

Isn't the fallacy of single blind testing the reason for the stories about Clever Hans, the "Talking Horse"?

>If the claim is that DBT is much more precise (and presumeably more sensitive in difference detection) now than it was even a few years ago,

The best tests of today take many more variables into account, and work harder to enable the listener to hear differences.

>then perhaps the next HE2006 event should be (instead of a debate) a DBT challenge in which the best set of professional (or other) listeners that John Atkinson can assemble participates in the best DBT's that each of JA and AK can create (in an off-site dedicated listening room, not with show background noise) to see if a significant difference is detected between, for example, a $200 integrated amplifier, a NAD 370, and a Halcro dm10-dm68 set up.

I would expect null results from all of those comparisons.

Previous post from me. Forgot to login.

Quote:
Both. If you believe that differences do not exist, that the differences are in my mind, I am not sure what you have to gain by converting me to your position, other than the satisfaction that you have done so. In other words, saving me from myself, which is what religious fanatics have demonstrated a need to do. I promise you that you are not in my will, and every penny saved from not purchasing audio equipment will not be left to you. So, there being no pecuniary interest in your success to convert me and others, we are only left with either your need to be "right", and proving to me your rightness, or the need to save me from myself, or the need to save me from charlatans (read: the devil). Either way, the similarities between you and the religious zealouts are striking.

Many valid points are raised in the above quote, explicitly and implicitly. The DBT-opponent views this as a debate between two "beliefs", where someone with a different "belief" is attempting to convert him against his will. He notes his obvious right to his own beliefs and the futility in attempting to convert him, all of which can be evidenced in the countless DBT-debates over the years.

This is why it is such a hot button. The DBT-opponent sees his beliefs challenged. His "religion" and self image are being questioned and attacked. He refuses the notion that taking a scientific and technical approach to evaluating the performance of technology is anything more than another "belief". He is basically degarding engineering knowledge and scientific method to having no more value and relevance than subjective opinion. The science that went into creating the audio component is no longer relevant when it comes to verifying if it is, say, audibly different from another component. Anyone who claims otherwise is a zealot...

OK, so we know why DBT is such a hot button. It might be interesting to explore why so many people (DBT-opponents) manage to develop "beliefs" regarding engineering products. Beliefs so strong that engineering can no longer be allowed to analyse the engineering products...

Furthermore, the question raised in the above quote, and implicit in the whole phenomenon, is why bother to reason and debate this matter? The DBT-opponents are happy and clearly state that they do not wish to have their beliefs challenged.

Let us view this whole matter from the perspective of the typical audio equipment customer. The "typical customer" (let's call him "TC") has no strong views on DBT and, more importantly, the price-performance benefits typically challenged by DBT resluts. He has most likely never even heard of DBT. He just wishes the most "bang for his buck". Where does he go for information?

- Audio magazines

- Audio dealers

- Equipment manufacturers (brochures, advertising)

These are by far the most visible and available sources of information to TC. Now let's take a cherished DBT-debate item: cables!

TC will most probably hear from his dealer(s) that cables make a difference and that more expensive is better. The dealer will "help" TC in putting together a well matched system where the cables match the price and performance of the rest of the system. Depending on dealer this means a cable budget of between 5 and 20 per cent of total system cost. (In my case, it was about 15 per cent of cool looking AudioQuest gear, with which I would presently gladly strangle my friendly dealer...)

TC will also have read some audio magazines, he might even be a frequent reader. In these magazines, he will most certainly have read/seen several tests of various cables. He will have seen the glossy ads of cable manufacturers.

It is to be expected that TC by now assumes that cables make an audible difference and that more expensive means better. It appears to be a common sense conclusion, right? Other people would not be buying expensive cables if they did not work. This becomes so obvious, that it is unlikely that TC even considers the notion that maybe cables are nonsense or that there is even a debate about their benefits.

We can therefore assume (and reality confirms) that TC will put some of his budget towards cables.

Now, if TC knew the following before making his purchase, he might act differently:

1. The value of cables is highly challenged and controversial within the audio community

2. There exists no technical reason or explanation as to why the performance of a standard speaker cable can be audibly improved upon since a cable is only transporting a signal and does so completely transparently (if properly specified and dimensioned).

3. People have NEVER been able to detect differences between cables in DBT's.

4. People who claim that differences exist have either failed to prove so in DBT's or refuse to participate in DBT's.

5. James Randi has a million bucks waiting for the first person who manages to hear a difference between cables.

6. Profit margins on cables are several times higher than on most other audio components. Thus cables are key earners for dealers.

7. Magazines can not exist without advertisers (who typically are either dealers or manufacturers reliant upon the profitability of dealers) and can therefore hardly question one of the key profit centers of the audio business.

Unfortunately, none of this info is typically available to TC, so he spends on cables. Depending on TC's personality, he is now becoming more or less unlikely to be open to admitting to having made a mistake. Placebo, that well known thing ALL us humans are subject to, is likely to make him "hear" improvements should he decide to compare with lesser cables. He might even with time morph into a true "cable believer" ("CB"), publicly defending his investment and new found belief.

THIS (I believe) is the reason why I and others bother to join the DBT-debate from time to time. Not to convert CB, but to give TC a fairer shot (in the admittedly rather unlikely event that he ever finds these threads and bothers to sift through the emotions and insults...). Sadly, the voice of reason is rarely audible above the roar of the industry. In audio terms: pro-DBT attempts to improve the signal-to-noice ratio on this very un-balanced information situation. With several million dB marketing hype, such attempts are probably as meaningful as expensive cables but at least they are cheaper...

To summarise and return to the thread question "Why is DBT such a hot button?":

- The DBT-opponents see themselves and their beliefs challenged and fight a battle of "freedom of belief".

- The DBT-proponents fight a frustrating and futile battle against the hype and lies of an entire industry.

The two end up fighting each other, when they are (often/probably?) fighting for two different and equally worthy (though somewhat misapplied) causes: Freedom of belief on one side and honesty and fairness on the other.

Just my two Euro cents...

Regards

In addition to subtle differences, there are many examples of irritating flaws in equipment that I've bought from well-respected and established companies, e.g.

- Failure to switch between tracks on certain discs (certain CD-r and certain copy-protected discs). Marantz CD6000. Very annoying.
- Failure to lock on to pre-emphazised CD even to the led is indicating it (certain CD only, Musical Fidelity X-24K). In addition, as measured by Stereophile, many players don't play pre-emphasized discs with correct frequency response.
- Failure to lock to external DACs (a Rotel player)
- Annoying mechanical noise (e.g. high-frequent mechanical noise on DVD-player Toshiba SD220E, that made me keep the Marantz CD6000 for CD-playback)
- Failure to give correct frequency response on digital output (no-name DVD player, -3 dB at 4.5 kHz). Reported from another person in Sweden.
- High and audible levels of hiss/noise. Marantz SR4300 HT amp.
- Failure to simulate center channel on DTS material. H/K HT amps. Irritating and very audible.
- Generation of audible noise by using so-called "AC filters".

Quote:

Quote:
To explain my quarter-century-old Damascene experience, you have to accept that either the blind test was flawed -- in which case all the reports that cited that 1978 test as "proving" the amplifiers sounded the same were wrong -- or that the nonaudio factors were irrelevant, in which case the criticisms of sighted listening based on that factor must be wrong.

Not to be too impertinent, but another possibility is that in 1978, you were not able to tell differences between amplifiers, period, and that over time nonaudio factors degraded your experience with your Quad amplifier once you were in a nonblinded situation.

Except, as I explained in my July 2005 article, the non-audio factors should have worked in the Quad's favor And my long-term dissatisfaction with the Quad was real. Continuing to listen to a poor-soundind system because of faith in the previous blind test results strikes me as delusional. This is why I instruct Stereophile's writers: "Report what you hear, _not_ what you think you ought to hear."

John Atkinson
Editor, Stereophile

Good topic to bring the traffic over here from AA, JA

The first problem with most blind testing (and all out-of-home testing) is time. Any time we are asked to sit down and make a judgment in a given time, we are at odds with our best intentions. At home, it takes somewhere between one hour and six months to figure out if we like something. In an AB test, we are asked to do it in a day or less and often in front of others (who invariably make noise, odd positive or negative body motions, etc.). The first clues to the failure of AB (blind) testing is in allotted time, unfamiliar location and company.

When we do blind testing, there are really two questions at hand: a) can we hear a difference? b) which one is better?

A couple of friends and I do blind testing at speaker events. We have found that most people, even people who are known "golden ears", cannot hear hear differences when put in a test environment. What is interesting is that when we remove the blind part, they can hear a difference. So if one AB tests knowing B is zip cord, almost everyone thinks they hear a difference (and I believe they do). The moment the test goes blind, they loose that confidence. So what it is about the "what if" that uses a different part of the brain than the "how different"? This might simply that audible memory is very short term, regardless of training.

At one event, we decided to test speaker cable. We found in the early rounds we would have to make the test very easy to get any result at all. So we collected some victims and moved the "bad" wire from 12ga to 14ga to 16ga to 18ga until we got a reasonable pass rate. In the show environment, 16ga and 18ga were the only wires to get any results that did not look random. But then we asked; "which is better?" Again, in our easy test when AB was known, better was clear. But even if the listener knew in test 1 that zip was B, the moment we pressed the randomizer, most listeners got lost. As soon as someone got it wrong, the debate began about synergy and how in some systems, perhaps zip cord could sound better or some part of our test was flawed.

Another thing about testing is how it blurrs the constant. This spring we were ABing some speakers. Both of these sounded good in earlier listening sessions, although they sounded different. Once they were side-by side, they sounded very different and both wrong. The mellower speaker made the brighter speaker sound very harsh while the colder speaker made the mellow speaker sound muffled. In this case, both measured reasonably well. The most interesting thing was that both sounded wrong in the AB environment.

What we were missing was not a scientist, but a psychiatrist. If someone hears a difference when they expect to, they congratulate themselves as all-knowing and golden ears. When they fail the test, there are a million reasons for the failure. In this business, right, or fastest, or most powerful, is limited to mostly subjective preference. We know that, but it still bugs us when we cannot prove our mettle in a public forum.

P

>We have found that most people, even people who are known "golden ears", cannot hear hear differences when put in a test environment. What is interesting is that when we remove the blind part, they can hear a difference.

DBT fans will have a field day with that pair of sentences, but I have a different comment: In some cases, even if there is not an audible difference under sighted (non-blind) conditions, one component will, over time, cause the listener to turn off the system in vague irritation, while another component may encourage continued listening. Perhaps sound not only affects our conscious processes, but also subconscious brain areas. That subconsious process (if it exists) may help form preferences, but will not be detectable in DBT as it is now conducted. How would we test the hypothesis that there are subconscious preference influences at work? That re-examination of testing might also improve the general testing sensitivity beyond current DBT thresholds.

>>We have found that most people, even people who are known "golden ears", cannot hear hear differences when put in a test environment. What is interesting is that when we remove the blind part, they can hear a difference.

>DBT fans will have a field day with that pair of sentences, but I have a different comment: In some cases, even if there is not an audible difference under sighted (non-blind) conditions, one component will, over time, cause the listener to turn off the system in vague irritation, while another component may encourage continued listening. Perhaps sound not only affects our conscious processes, but also subconscious brain areas. That subconsious process (if it exists) may help form preferences, but will not be detectable in DBT as it is now conducted.

Here we go again - the "DBT can't be long term" myth rides again!

> How would we test the hypothesis that there are subconscious preference influences at work? That re-examination of testing might also improve the general testing sensitivity beyond current DBT thresholds.

In the end, conventional listening tests whether sighted or blind, rely on some kind of conscious perception by some body by some means.

There is no reason any or all of that can't be done under bias-controlled conditions.

"Here we go again - the "DBT can't be long term" myth rides again!"

I don't know this myth. I only know that in most cases, blind or not, a group of people are asked to spend some time, usually short, in a new environment and evaluate something. Not a good situation.

"there is no reason any or all of that can't be done under bias-controlled conditions. "

Can you name a "bias-controlled condition"? A cannot, especially since I carry my biases around with me. I already know your system does not sound as good as (insert name here) because I already have feelings about this and your stance. The same is true of you. Suggesting you (or anyone) can overcome or ignore bias is simply lying to yourself.

Psychotherapy is paying someone to listen to you avoid problems you admit having.

P