| Columns Retired Columns & Blogs |
December 13, 2008 - 7:23pm
#1
Double Deaf
| Loudspeakers Amplification Digital Sources | |
Analog Sources Accessories Featured | |
Music Columns Retired Columns | |
Show Reports | |
Features Latest News Community | |
Resources Subscriptions |
© 2024 StereophileAVTech Media Americas Inc., USA
All rights reserved
uh yeah, many times. Hence me not buying any of the fringe products any more...
I've never attended a formal double blind test but my friend who builds amps has often put me to the test by changing components, or not changing them, in his amps I've been running. So far I've correctly identified 15 out of 21 correctly and imagined I heard improvements 3 times when he'd changed nothing. Was I embarrassed? No. Will I still let him fiddle with my gear? Yes.
If the test was well-designed and valid, then I would agree that I could not distinguish between what I may have thought I could before the test.
Many of the tweaks I have installed in my own audio system were not verified by blind testing. If the changes were obvious enough, I don't bother with blind tests, as the difference is much more than placebo effect could cause.
If the difference is subtle, then I'm not sure the difference is real without doing a blind test. Usually such a test was not done, because of inconvenience, difficult to do the test properly, etc. Such a test may help eliminate the placebo effect from the detection of differences.
In the Stereophile Test CD (STPH-002-2), there were two tracks #16 and #17, recorded using two different digital transfers. On track #18, the same piece is played but assembled from both A/D converters, 10 times ("edit points" as the CD insert says). I and others listening could tell that track #16 sounds different from track #17. I have only tried this once, but could not tell where the 10 edit points are on track #18. I think that with lots of practice listening to the tracks #16 and #17, I should be able to tell the difference in #18.
I am really intrigued by the blind testing phenomenon. I don't mind when I fail - it helps with the journey!
I enjoy having a friend or the wife change something and try to see if I can hear the difference.
Absolue polarity is a fun one, and can be surprisingly evident.
I've run across cables that I can tell reliably, too.
I think the best place to start investigating is with any of your gear that has variable controls of any kind.
A CD player with variable sampling rates, polarity (as I mentioned,) even trying to trick yourself with changing speaker phase - all just to get yourself into 'critical mode' where you are actively listening for differences.
I've mentioned it before, but something I find fun is listening very very near field and finding a difference, then seeing how far back from the speaker I can go and still be able to tell that difference.
If Double Blind Deafness is real, then nothing wrong with exploring it and trying to explore the limits.
Heck, we all have old crap cable or speaker wire around. Great tools for learning to relax in a blind listening session!
Cheers. Hope that wasn't too pedantic.
OK, how about an old blind favorite: CD timed to match the LP, and switching between the two via an intermediary?
Easy to tell, but which do you prefer?
I have a really quiet pressing of (
) Sweetest Taboo that I used on my wife (then girlfriend) to demo the differences, and she's been into vinyl ever since! 
Now, all I gotta do is sit and listen and she switches between the two in real time. Much fun, and it helps with cartridge comparisons, too!
That's intriguing. I don't think I've ever A/B'd LP and CD in a direct shoot-out. I'll drag out some "dupes" and do a comparo. I'll try to match levels as best I can to try to compensate for phono/line level sensitivity. I really do think that a good LP sounds pretty terrific, though. (Even some of those old Dyna-Grooves)!
It would have to be a DBT that I believed in...No 10 minute sound bites of unfamiliar music on an unfamiliar system but a months long test on my own gear with my own well known music.
I have no idea how to set up something like that. Anything else would result in what I expect, my failure to recognize minor changes in unknown systems of unknown music.
Strictly speaking, you are certainly right. But this test would be done in an informal context, as (I think) Buddah alludes to, even to the point of knowing what source is playing at any given point in time, only as a means to describe relative differences between the two sources playing identical program material. In other words, a "just for funsies" kind of thing.
Your endorsement of a DBT protocol for serious comparisons is one that I completely share with you.
OK, I gotta ask. This is Hi Fi, what on Earth would require "serious comparison?"
I did this on several occasions. It depends on the quality of the CD or the LP, in which I compared the Sheffield direct to disc vs. the CD, of the same performance (take). Unfortunately, record wear may bias the results in favor of the CD.
A good DBT does not depend on familiar systems or software. An important part of the test is subject naivete. You don't know which is which or what is "good", only whether you can reliably identify a difference between two choices.
WTL, given all the parameters involved, wouldn't you think a DBT on your own system would be either more precise, or at least more efficient?
I think that familiarizing oneself with the sound of a given system in a given room is paramount for being able to more readily identify differences.
That would sort of undermine the whole thing, I believe. an audible difference would be apparent no matter the location. the more "favorable" conditions/caveats you attach to it, the less authentic the testing is..
as I said, if there are audible differences, they will be very apparent.. it shouldnt be "well, if I am in my room with my favorite blanket with the heat on 72.3 and my favorite underwear on I can hear the difference"...
Nope would not bother me at all. If I may suggest, try breaking the test down into several sessions of 3 ABs (back and forths), making the comparison more real world. That way you will at least minimize "Habituation to Stimuli" effects.
For those who are not familiar with "Habituation to Stimuli" in audio terms, in a general sense, it is an automatic unconscious response/being desensitized that causes two close but different sounds eventually being perceived as sounding the same. From reports that I hear, and my own testing, this take a few ABs, 4 or 5, before the effects are clearly noticed.
Hope this helps. Take care.
Here's one of those things(post with information) That your subconscious will do everything in it's power to forget, ridicule, ignore or lambaste the messenger:
-Cutting edge physics is now saying that ALL 'reality' is VIRTUAL.
And, that it is both 'real' and consensus in nature, and 30% of that reality is consensus in nature, ie all of us deciding it together. The rest..the other 70% is just a rolling roiling resonant foam wave that we exist on/in....riding the edge of.
That 30%? With enough knowledge, will, and understanding ---we can manipulate it.
Thus a room full of naysayers can collectively will/pressure you to not hear difference(s) you >>KNOW<< are there.
Or the test can be devised by folks who do not know where the subtle differences are coming from and thus create an invalid test. Happens every day.
If they didn't insist on a 'test' to prove that you can hear it, they would know already where these differences came from, and how those differences were enacted, carried and shifted from one component to another. Thus, their silly asses are nearly 100% capable of creating an invalid test. And usually do.
Then their ignorant selves find a way to blame their ignorance on you.
Nothing new under the sun.
Thanks for the info KBK.
what info?? it was a circle jerk, a massage and an "atta boy" for the voodoo crew here... nothing new under the sun indeed.
I haven't taken part in an actual formal DBT, although I have tried using the computer audio player Foobar2000's ABX facility to ABX absolute polarity in my computer audio setup. I couldn't identify it.
Though it wasn't a formal DBT, I did take part in an informal amplifier comparison at the 2006 RMAF run by Bob Cordell. It's described in this article in AudioXpress magazine. He had a 35W tube amp and a 200W solid-state unit hooked up via a switching arrangement, where one could switch back and forth between the two with a hand-held switch with positions labeled "1" and "2". We tried to informally guess which was which. Levels were matched to better than 0.1 dB. It was hard! The constantly-changing nature of the music I used to try to identify the amps made it so that when I thought I had a difference nailed down, the music would head off in another direction and I would lose my frame of reference.
If I had it to do over again, I would have brought some very repetitive music to make this easier. I was trying to do it by identifying "flabby bass" in the tube amp due to lower damping factor relative to the solid-state unit.
I'm not one of the "golden ear brigade", so I had no big ego investment in trying to identify one from the other. It was definitely a thought-provoking experiment. It certainly made me suspicious of the exaggerated "night and day" claims I've often read in the audio press. Of course, if such supposed differences didn't exist, they wouldn't have much to write about!
I agree.
Buddha, if you can detect a difference with your system and cannot with an unfamiliar system (or vice-versa, and assuming both tests are done well), then the your detection is not reproducible, which is one of the conditions to have an experiment's results accepted as valid. Then you would have to examine which test was good and which bad, or what else might have caused those results.
I still disagree.
I think it takes more time to settle in to listening for differences in an unfamiliar room.
A listener may learn to take cues from his system that may more readily point out differences.
I'm interested in the best situation possible to try and identify differences.
I've 'learned' alot about how my system does certain things in my own room. A new room with unfamiliar gear would not facilitate as efficient a process of identifying differences.
Same goes for recordings. Familiarity with what's on a CD or LP makes testing more efficient.
With your limitation on testing, at the same time as you are expected to identify small differences, you don't even know if the room, speakers, etc...properly reproduce what nuance you might usually listen for.
If we are trying to change only one variable at a time, then I'd say to do it at home, not some place where all the variables are new.
We're probably splitting hairs, I like tests where I am familiar with everything, and I change only one thing. If I could not do that, I'd still play, but as someone above mentioned, then I'd want lots of time to get used to the whole room/system/AC ducts/traffic etc...change!
Quote: "...if you can detect a difference with your system and cannot with an unfamiliar system (or vice-versa, and assuming both tests are done well), then the your detection is not reproducible..."
Let's say I knock out a DBT on my system with a cable. 10 out of 10, seven days in a row.
Is that considered not reproduceable?
It may not be 'generalizable,' but it is definitely reproduceable.
Now, if I claim my finding will be reproduceable on all systems, then I may be wrong. Perhaps the finding was better delineated on gear with certain impedances, etc...
My finding may be system dependent, but it would be obviously reproduceable.
Buddha, if one cannot hear a difference in any system, any time, with any piece of music, then said difference is not worth mentioning. familiarization makes the whole thing moot.
Inconceivable!
Are you thinking all systems share the same level of resolution and frequency response?
Noise, distortion...all the same?
If you can repeatably hear blind what something does in your system, but on a generic system you did not, then there is no value to the DBT result on your own gear and the result is moot?
Do you shop that way?
Seriously, "in any system, any time, any piece of music?"
Have you never heard a crap system?
Have you experienced pieces of music that may be more favorable to demonstrating bass response than others?
Well, at least you guys are going to make Ethan happy.
I will remain in dissent.
I'm with buddha on this one.
Are you guys serious ?
But can anyone? I doubt it. We are not talking about the difference between AM radio and a CD, we are speaking of subtle changes in VERY familiar systems. If one is not familiar with the system and the music, the size of the change needed to hear it grows exponentially.
The problem with DBT's is they make a false 'either/or' comparison. Either one can distinguish a change in someone else's system A in a short period of time reliably or there is NO change. I believe the extent of the change is directly related to the familiarity one has with the system, the music and time.
The better you know the system, the more likely one is to hear a change.
buddha, obviously I dont mean a boombox from walmart... but any decent system.
and I dont give a shit about making ethan happy, my opinions are mine alone.
and as far as being in dissent, hell, I aint mad atya..
it takes all kinds, but my experiences recording,playing, singing, and listening to music (singing and recording professionally) for much of my life have led me to the above conclusions. Its all good though, so cool.
.."The better you know the system, the more likely one is to hear a change".
...or not hear a change. That's why it's a perfectly reasonable condition that a familiar setup (ie., your own) be used as a starting point if you wish to compare different components (one at a time, of course).
I think that any reasonably astute listener could hear a difference between two completely different systems, especially if the speakers are changed along with everything else.
It is not hearing differences between different systems but hearing a subtle difference on an unfamiliar system that is the problem with DBT as most folk understand it. It is not that such a test has no value but, IMHO, that subtle changes in an unfamiliar system with unfamiliar music might really exist but not be heard by the DB testers.
To hear a real change you must know the system and music cold and listen over a lot of time. If someone could figure a way to do that, the DBT might be of value. As it is done, I find it of little value as it does not prove the existence of anything but the most grand changes. Potential real changes are simply missed. As they are missed devotes of such tests claim there is no change and the claims made about such gear is snake oil.
I fault the test procedure itself.
As I have posted. I have changed cables and heard no change on my system for days or sometimes weeks BUT, when I get around to a specific CD, something different appears. To me that makes for a real change but a very subtle one, one that would not be ever seen in a DBT.
I am of the belief , that, if it were not popular audiophile opinion that cables make some sort of difference, had you never read that, heard it from anyone, etc... you would hear no difference. (the exception being cases in which a cable was physically damaged and you swapped it for one that was not)
And if some wondering genius had not discovered that distilling potatoes would lead to a magic new drink we would not have Vodka. Does that means Vodka or the potential for it in potatoes would not exist.
I like your correlation. What perplexes me about DBT advocacy is the very unscientific denial that "controlled" parameters imposed on the multiple sensory variables of the participants is supposed to produce repeatable, empirical results. Haven't the historical failures of behavioral psychology (i.e. Skinner, et al) discredited the co-opting of laboratory protocols applied to human subjects? We are NOT rats (at least most of us)and our respones to music and sound differences would seem to be better served by more holistic means than the inept aplication of reductive DBT's.
Regardless of where we stand on this well-worn issue, I wish everyone a Merry Christmas, Chanukha and/or seasonal celebration of choice! And much good music to all!
And a merry Christmas to you as well!
Buddha, I understand your position. I've been there as have many of us in this forum.
What a proper DBT helps achieve is the elimination of the observer's bias, a fact I'm sure you well know. If you need to settle in to hear differences, then I'm not sure what that implies. But if you need "cues" from your own system to tell any difference, then it could be argued that those cues are an uncontrolled variable.
The "best situation" as you said, is desirable for the test. A DBT test which removes the observer's bias, and concentrates on whether a difference is detectable by the observer under those conditions, does not need a familiar system, only one the is "good enough" (whatever that takes) not to cloud the difference being tested, and remains the same for the A vs, B conditions of the test.
One is not listening whether A is better or worse than B, only if there is a detectable difference. If no difference is detected, it does not mean that no difference exists. It does mean that either the test is not sensitive enough or the difference is not detected under these test conditions, if there is a real difference. Generally speaking, test design is difficult to do correctly. Not all the variables that matter are completely or well controlled, leads to a false conclusion, which really is the result of a bad test. It does not invalidate the DBT principle. If you look from the DBT side, a sighted listening test is an uncontrolled test, therefore likely biased.
In a well-designed test, the only variable is the A vs. B under test (device, cable, etc.); the rest of the system remains the same, therefore only one variable changes (A vs. B). That's why it does not matter using a different system, only one that is capable enough for the difference to be tested.
You're correct, in part. What you would have demonstrated on your own system is repeatability, not reproducibility. (terminology - I know what you mean)
For example, if I'm convinced that my capacitor mods give the sound a different character that I can describe (I judged it as an improvement), and it was missing on my system prior to the mods because of my familiarity with my system, I still cannot claim an absence of bias without a good DBT. Yet I still think that the difference is real because I can describe the difference, clear to me based upon my musical training. This is like what you are saying, if I understand it correctly.
Now if I take another system (maybe someone else's) and listen to it before the mods and again after the mods, and if I hear the same type of improvement in the sound, then I would think that these mods "work" and can be applied in the same way probably on other systems too. That would be reproducibility, from one system to another.
Of course, it's possible that a given system may not be good enough and the improvement is not heard because some other fault is supressing the detection of the improvement. That's a different ball game, like modding a two transistor AM radio (assumption made that cap mods do not improve sound here). If those faults are removed, and the same improvement can be described, then the effect is reproduced on that system too.
The best way to test those mods is two systems that are judged to sound the same, perhaps under DBT, again to remove bias. Modify one of the two and compare, under both sighted testing and DBT.
We should understand that DBT is a method that helps remove bias, among other things. If you hear a difference on your system, how do you convince others that you're not "hearing things"? I asked myself that same question after making mods to my system. The more I listened, the more convinced I got that the improvement is real. But, I have not eliminated my own bias, i.e. I could be fooling myself and not know it!