| Columns Retired Columns & Blogs |
January 18, 2008 - 12:50pm
#1
comments on blind testing
| Loudspeakers Amplification Digital Sources | |
Analog Sources Accessories Featured | |
Music Columns Retired Columns | |
Show Reports | |
Features Latest News Community | |
Resources Subscriptions |
© 2024 StereophileAVTech Media Americas Inc., USA
All rights reserved
In addition to misunderstanding the comment you cite, this dead horse has already been so thoroughly pulverized there isn't even a grease spot left in the pasture where he used to lie.
Some sites ban DBT talk as it typically ends with the hurling of insults rather than any sort of illumination.
No, I don't think I misunderstood JA's comment, nor do I think the subject of DBT should be off-limits simply because it causes some people to be insulting.
JA wrote on p. 10 that a blind comparison test may find no difference either because there isn't one, or because the blind test "may not have the resolving power to detect a small but real difference." True, but let's go a little farther:
John's comment begs the question that follows from Geoffrey Hudson's letter about placebo effects: "OK, if there is a difference between A and B, then how do you know whether it's real but small, or whether something else other than a real difference (e.g., a placebo effect, or chance) caused you to think you heard a difference when none exists?"
My comment was aimed at pointing out the easiest way to increase the "resolving power" of blind tests, given adequate listening conditions. Please clarify (and let's stay civil) what you think I misunderstood.
SBT, DBT, ABX, ETC. have all been hashed to death by audiophiles; for example, starting over twenty years ago in Stereophile.
Unfortunately, most of the participants in forums know little of biostatistical protocols or analysis, let alone even basic design. Thus, the discussions are essentially expressions of bias founded in assumption, backed up with speculation.
There are many excellent sources of information on the subject, including employing various forms of blind testing in sensory analysis if you are truly interested.
Happy researching.
While statistics do have their uses, having a larger listener base is completly irrelavent to me.
Statistically, based on a very large base MP3 sounds just like a CD and the Bose Wave Radio sounds just like a music hall.
When comparing equipment I only care about a sample base of one. Me.
I feel the same way. My purchases have to appeal to my ears. It doesn't matter what anyone else thinks. Don't we all like to think we're immune to having our opinions influenced by other people? Unfortunately, 100 years of social psychology research indicates that many people can be influenced by the opinions of others without being aware of having been influenced.
Reviewers have some influence over the choices some buyers make--look at the posts here as well as some of the Letters in Stereophile. So anything they can do to eliminate placebo and observer effects would be an asset to our hobby.
In the last year, Stereophile has shown a subtle shift in its position on the validity of blind listening.
While still seemingly holding to its official editorial stance that blind listening is not valid as an audio tool, JA wrote in one month's "As We See It" about being part of an audience that could hear differences during a blinded listening session of different digital formats. MF wrote in one of his columns about the Continuum turntable being demonstrably superior to other tables as evidenced by subjects reporting the same while listening under blind conditions to a CD-R. Just this past week, JA bragged in his CES coverage about 'passing' a one trial blind test of cable differences.
I think Stereophile has shifted position into now allowing reportage of instances in which a difference was detected.
I mean, if a one trial success story is now legitimate enough to be newsworthy, then the future of blind listening test reports in Stereophile should be rosy as hell!
For now, I'm sure negative or null resluts will remain anathema, but at least positive examples are now being acknowledged. That's progress, eh?
(Pardon me, I'm just stirring the pot.)
Interesting observation Buddha, and one I agree with too. I think the shift is that these tests are ones that are more relaxed, play real music and are closer to real listening conditions. They use good equipment and then change one factor, but allow for listeners to get a feel for the sound in each case. I'm on board with this being at least one aspect of equipment reporting, although honestly I trust most reviewers to not need it most times. I know their tastes in general and can take that into account the same as I would a film critic. I used to read this one critic all the time who almost never liked the movies I liked, but he was consistent and I could sort of take his reviews for what they were anyway.
To the extent that blind testing can be done that is valid and reliable (for our objective side) and USEFUL and ILLUSTRATIVE (for our subjective side), then yes, please bring it on. Yep, easier said than done.
Reviewers do have impact on us - and generally we consumers do NOT have the chance to do nearly as much comparative listening as the reviewers. Therefore it is reasonable that we give their opinions some weight. So, if the reviewers get supported (or not) in blind tests, that helps us consumers, a lot.