>Why is blind testing not such a hot button with:

- Drug testing in the pharmaceutical field?

- Wine testing?

- Perfume testing?

- Food testing?

>Where lies the difference(s) between these (and many other) areas where blind testing is common and non-controversial and the audio field?

In a word: pragmatism.

Quote:

Why is blind testing not such a hot button with:

- Drug testing in the pharmaceutical field?

- Wine testing?

- Perfume testing?

- Food testing?

Where lies the difference(s) between these (and many other) areas where blind testing is common and non-controversial and the audio field?

I think it comes down to the fact that in all those fields, what is being tested is the direct effect of the stimulus. With audio, you must test the stimulus indirectly, through its effect on music, which itself has a varying effect on the listener. (If it doesn't, it isn'ty music!)

I think this is why blind tests of audio components are much more sensitive using test tones than music. But then, the fact that test tones are not music removes the test one step from reality.

John Atkinson
Editor, Stereophile

Quote:

I think it comes down to the fact that in all those fields, what is being tested is the direct effect of the stimulus. With audio, you must test the stimulus indirectly, through its effect on music, which itself has a varying effect on the listener. (If it doesn't, it isn'ty music!)

Not sure how to read this.

1. The same music has varying effects when it is played back on various components? If so, why do DBT's not detect these differences?

or

2. Music is such a stimuli in itself that it over-shadows differences between components? If so, why is sighted testing more valid where the placebo effects would be added to the already strong stimuli from the music? Or for that matter, why is any form of testing relevant or even possible?

or

3. Something I'm missing?

I understand the difference in distinction between indirect and direct effect on the stimulus, but fail to see how it matters with regard to DBT or why it would explain the strong emotions around DBT unique to the audio field.

Regards

Also, in audio, we have a reference: live music. I have to consider this a little more, but I don't think wine belongs on that list.

P

Hi.

>I think it comes down to the fact that in all those fields, what is being tested is the direct effect of the stimulus.

Certainly not true in the realm of medicine.

>With audio, you must test the stimulus indirectly, through its effect on music, which itself has a varying effect on the listener. (If it doesn't, it isn'ty music!)

At this point the topic is all of audio.

>I think this is why blind tests of audio components are much more sensitive using test tones than music.

It is not true at all that listening tests are always more sensitive when performed with test tones.

Note also that without any introduction support, Atkinson has changed the topic from audio tests in general to just blind audio tests.

It is true that some kinds of listening tests involving pure tones can produce more sensitive results than music. However, its not a general rule. These effects are observed with sighted or blind tests, only with the sighted tests its harder to tell whether people are hearing something, or reacting to their biases.

> But then, the fact that test tones are not music removes the test one step from reality.

Ah, the punch line - the message from Atkinson is that somehow blind tests are one step removed from reality. His reality, perhaps, but not everybody's reality.

To summarize there are two false claims here. One is that listening tests done with test tones are always more sensitive, and the other that this effect is stronger in blind tests than other tests.

Quote:

Quote:

I think it comes down to the fact that in all those fields, what is being tested is the direct effect of the stimulus. With audio, you must test the stimulus indirectly, through its effect on music, which itself has a varying effect on the listener. (If it doesn't, it isn't music!)

Not sure how to read this....I understand the difference in distinction between indirect and direct effect on the stimulus, but fail to see how it matters with regard to DBT or why it would explain the strong emotions around DBT unique to the audio field.

With, say, wine tests, the chemicals in the wine interact directly with receptors in the mouth and nose, and it is the effects of those interactions that are being tested for. With audio, the changes under test affect the behavior of the playback equipment on the reproduction of audio signals, and for stereo playback, we don't experience those signals directly but only how they affect our interpretation of the music carried by those signals, which itself is an illusion.

For example, there is no such thing as a stereo image. The image most of us so readily perceive from stereo playback is entirely a mental construct. When performing tests on audio components rather than, say, wine, we are comparing illusions rather than direct experience, which is why testing becomes more difficult, less conclusive.

John Atkinson
Editor, Stereophile

>>>Why is blind testing not such a hot button with:
>>>- Drug testing in the pharmaceutical field?
>>>- Wine testing?
>>>- Perfume testing?
>>>- Food testing?
>>
>>I think it comes down to the fact that in all those
>>fields, what is being tested is the direct effect of the
>>stimulus.
>
><snip of perheripheral comment>
>
>>With audio, you must test the stimulus indirectly, through
>>its effect on music, which itself has a varying effect on
>>the listener. (If it doesn't, it isn't music!)I think this
>>is why blind tests of audio components are much more
>>sensitive using test tones than music.
>
> It is not true at all that listening tests are always more
> sensitive when performed with test tones.

This has been my experience, Mr. Krueger. For example, using
test tones I can and have detected a reversal of polarity
100% of the time using an ABX box. With a music source, even
where the recording's absolute polarity is known, my scoring
drops.

> Note also that without any introduction support, Atkinson
> has changed the topic from audio tests in general to just
> blind audio tests.

The original question specifically mentioned _blind_ tests,
Mr. Krueger, and it was that to which I was responding: "Why
is blind testing not such a hot button?" the poster asked.

>It is true that some kinds of listening tests involving
>pure tones can produce more sensitive results than music.
>However, its not a general rule.

As I said, my experience has been that it is generally
correct. I would be interested in hearing of exceptions.

>> But then, the fact that test tones are not music removes
>> the test one step from reality.
>
>Ah, the punch line - the message from Atkinson is that
>somehow blind tests are one step removed from reality. His
>reality, perhaps, but not everybody's reality.

See my previous message this morning for the reasoning
behind my statement, Mr. Krueger.

>To summarize there are two false claims here.

I request you please refrain from making blanket, accusatory
statement like this on this bulletin board, Mr. Krueger.

> One is that listening tests done with test tones are
> always more sensitive...

As I said, this has been my experience and that of others.
Again, if there are exceptions to this, I would be
interested in hearing what they are.

> and the other that this effect is stronger in blind tests
> than other tests.

I have not said this. Please refrain from putting words in
my mouth, Mr. Krueger.

John Atkinson
Editor, Stereophile

Could we then conclude that all terms usually being quoted to define how good a system is, is all based on an illusion?
If "hearing" the image or the focus is part of an illusion created (wich I certainly agree) while listening, we could increase the perception threshold by aiding the audition with some good malt...and have better illusions!

brgds, marcelo

>

Dear Mr Atkinson

Quote:

Quote:

Why is blind testing not such a hot button with:

- Drug testing in the pharmaceutical field?

- Wine testing?

- Perfume testing?

- Food testing?

Where lies the difference(s) between these (and many other) areas where blind testing is common and non-controversial and the audio field?

I think it comes down to the fact that in all those fields, what is being tested is the direct effect of the stimulus. With audio, you must test the stimulus indirectly, through its effect on music, which itself has a varying effect on the listener. (If it doesn't, it isn'ty music!)

I think this is why blind tests of audio components are much more sensitive using test tones than music. But then, the fact that test tones are not music removes the test one step from reality.

John Atkinson
Editor, Stereophile

The reason blind test are used in pharmaceutical field is NOT that it is direct stimulus!

Poeple tend to be well when they believe they are treated! With active drugs OR with suger-pills! The reason to use "blind-test" is to have an statistic contoll group not exposed to the active pharmaceutical that belive they are given active medicine. The pharmaceutical has to be better then the placebo-effect of non active treatment, which can be VERY big, even on severe deseases. The ONLY acceptet way to test this is by Blind-tests, as they do. Ask FDA in the US!

This meen, the belivef is strong! and we may hear a difference in audio that is not there when we know what is compared. This in line with the placebo effect that new drugs must beat.

A difference in open tests that can not be detected in blind-test are in pharmaceutical terms placebo-effects, nothing else!
The same should be valid for music, test-tones and sound in general or...?
To say that some has a higher moral and can in open test suppress the effect of placebo is not a valid argument for open tests instead of closed. The individs in pharmaceutical test know that they can get blank medicine but they still get cured.

In pharmaceutical we play with the health of the population in audio we only play with there money.......

/Magnus
(Ph. D. in Chemistry were I had close contact with the pharmaceutical industi)

Quote:
>

Hi.

What

While DBTs indubitably are employed in the pharmaceutical field, although recently that's come under fire, in the others:

- Wine testing

- Perfume testing

- Food testing

they are used rarely, and only for a goof, as it were. As when in France a group of French wines are judged alongside a group of Americans. Or, vice versa. National/regional loyalties might come into play.

Otherwise all the design and testing/tasting are done openly, because the participants honor their own and others' pallets, and who needs the rigamarole?

Speak to a professional coffee taster about what I just said, for confirmation.

clark

I don

To Arny or anyone else who can answer this question,

How does one design a DBT that tests a particular piece of audio equipment that will give a meaningful and significant result? For example, how do you factor out age, medical conditions, or any number of variables that can affect a listener's hearing?

>How does one design a DBT that tests a particular piece of audio equipment that will give a meaningful and significant result? For example, how do you factor out age, medical conditions, or any number of variables that can affect a listener's hearing?

The same way you do the same thing with sighted evaluations.

Quote:

>How does one design a DBT that tests a particular piece of audio equipment that will give a meaningful and significant result? For example, how do you factor out age, medical conditions, or any number of variables that can affect a listener's hearing?

The same way you do the same thing with sighted evaluations.

Ok...and how would you do that, ie. factor out age, etc. in a sighted evaluation?

So far, I have not heard anyone say or read anywhere on this forum anything to the effect of "I bought component XYZ by doing a DBT and boy, am I glad I did!".....

Even though the gang at UHF did a group test as well as not being "double blind", I stand corrected....

http://uhfmag.com/Newsletter.html

With all due respects anybody with an real training in research on human subjects has to like blind testing at some level. We are not Mr. Spock, our likes and decisions are not based entirely on logic and solid empirical data. We have prejuidices and they play out.

If a piece of gear starts out good looking, you approach it more favorably, Have it from a manufacturer that you've had good experience with and you start the audition in a much better frame of mind.

And that really matters. When I made my last purchase I was comparing gear that was uniformly very good, and well reviewed, including by this magazine. I'd have loved to have gotten everything together at the same SPL to really compare. It would have cut down on the variables that I had to weigh.

What audio is, and ought to be, about is the sound and the enjoyment of the listening experience. At the end of the day anything that distracts us from concentrating on the sound hurts. I think blind testing is a good idea where it is practical. To argue that we're 'too competent' to be influenced by our emotions is patent nonsense.

I can see where you would reach the conclusions you have given that you apparently hear little if any difference in superior cabling.

Many years ago, Jonathan Scull did an interview with Jack Renner of Telarc fame (Stereophile Vol.21 No.10) and got around to discussing how Renner achieved his particular sound. Part of the discussion, which is one of my favorite Stereophile interviews of all time, went something like this:

Scull: Is the mixer tweaked in anyway?
Renner: We have used hot-rodded consoles. We had one that was internally wired with Monster Cable that was used for a long time. In fact, a number of years ago we were recording the Cincinnati Pops and the Monster Cable console went down-nothing to do with the fact that it was wired with Monster Cable, of course.
Scull: Of course...
Renner: We actually use a combination of Monster Cables and MIT. We're good friends with both Noel Lee and Bruce Brisson.
Scull: Imagine...
Renner: So a component failed in the middle of a session. We had a backup console from the same manufacturer, Neotek, and it took us only 10 minutes to set it up. Now, understand, we've got a signal that's spent its whole life from the microphone output to the A/D going through Monster Cable. Then we switched to the console with the standard cable in it. And the minute I brought up the fader, everybody in the control room-not just my technical assistants and the producer, but the orchestra manager, the musicians who weren't in the particular number--they all said, "What did you do, what happened to the sound?" Everyone could hear the soundstage got smaller. Everything just got a little more narrow and not quite as bloomy. If you ever needed a demonstration of the effects of hig-performance cable, that was it.
End snip...

Cabling may be overpriced considering what it actually is, but suggesting that the audible differences are insignificant or even 'voodoo' as some say, is nonsense.

It can be shown quite easily that every individual person hears differently. "How" and "what" we hear can be affected by age, disease, medication/drugs, anatomy and physiology (eg. the bones in our ears or the shape of our outer ears and head) and anything that may alter the form and function of those anatomical parts (eg. physical trauma, brain injury), mood/emotions, perceptions, beliefs, preference, training, cultural background, and childhood upbringing. These are just some of the factors that come to mind. Without controlling or eliminating these individual variations in hearing by using very large sample sizes, as Anonymous above has suggested, or by using a very specfic testing group (eg, 45-50 year old men with good hearing and in good health, who live in large cities and make over $75,000 a year) then the results of even a well designed double blind audio test can only be valid for the individual listener being tested. A double blind audio test comparing different components that's properly designed and executed might yield some interesting information of some kind, but it's not a very practical way to shop.

I concur with Mr. Atkinson. Test tones cannot represent reality. Never heard no test tones at any live concert I ever went to.....come to think of it never heard much of anything else during and after one either.