Audio, Meet Science

For a field based on science, high-end audio has a relationship with its parent discipline that is regrettably complex. Even as they enjoy science's technological fruits, many audiophiles reject the very methods—scientific testing—that made possible audio in the home. That seems strange to me.

Of course, there are reasons—not necessarily good ones—why some people in audio have a low opinion of science. The entire hobby/industry suffered a grievous blow a generation or so ago, when too much focus was put on something—how an amplifier measured—that resembled science, and too little on how such products actually sounded. A little later, the Compact Disc's promise of technical "perfection" yielded dubious sonic benefits when compared with an older, simpler medium, the LP. And there has been, over the years, no shortage of self-identified scientific types manipulating purportedly scientific tests, or ignoring inconvenient test results, to support their preconceptions.

Subjectivists, meanwhile, sometimes seem to intentionally hold themselves up for ridicule. A few audio writers, especially for the online 'zines, seem eager to prostitute themselves for the latest preposterous product—the Intelligent Chip, the Shakti Hallograph Soundfield Optimizer, the Machina Dynamica Brilliant Pebbles, the Marigo Audio Lab Dots, the Tice Clock. Meanwhile, some prominent industry folks have consistently failed, in my opinion, to maintain a sufficiently skeptical posture toward such products. It doesn't help that in audio there has been a long tradition—a dogma, even—of "trusting your ears," despite abundant evidence from neuroscience and cognitive psychology that our ears and other senses, though supremely acute, are supremely unreliable.

Yet science—and scientific testing—has much to offer audio. Audiophiles ought to embrace scientific methods with the same healthy skepticism with which they embrace sighted, subjective equipment evaluations: as a tool that, though subject to misuse, is invaluable in the hands of honest people with the right set of skills.

One form of testing that's especially important—and especially controversial—is the use of rigorous methods to validate apparent perceptions: Did you really hear what you thought you heard? Is the sound really different with that new amplifier/cable/CD player from what it was with the old one? Does putting that photo in the freezer really change the way the system sounds?

Rigorous tests can offer scientific validation of subjective observations.(footnote 1) Such validation doesn't come easily, but when it comes it's a force for truth, justice, and the American way.

Such tests have two possible outcomes: Either you establish scientifically that there's a difference between A and B, or you fail to establish a difference. And here's a crucial point that's often overlooked by people on the objectivist side: While a positive result establishes the reality of a perception to a certain level of confidence, a null result—a failure to reliably detect a difference—does not indicate the nonexistence of that difference.

To cite an oft-quoted phrase that's sometimes attributed to Carl Sagan, "absence of evidence is not evidence of absence" (footnote 2). Or, to cite Les Leventhal in an AES paper, "Properly speaking, a statistical conclusion about H0 from a significance test cannot justify 'accepting' the scientific hypothesis that differences are inaudible" (footnote 3).

In simpler language: Differences we think we hear but that testing fails to validate may nonetheless be real. An experiment that fails to show a difference between the sounds produced by two amplifiers does not indicate that no audible difference exists.

Many of the rigorous listening tests that are relevant to audio are difficult and tiring to perform, requiring serious concentration, many repetitions, and sometimes heavy lifting. Things can be made easier by using many listeners at once, but then the only conclusions you can draw are about the average characteristics of the group. The group average may not permit distinguishing cable A from cable B, but that doesn't mean a particularly golden-eared member of the group can't.

One of the beautiful things about science is that often you can make your experiments more sensitive by applying new technologies. The likelihood of a non-null result can be squeezed and squeezed until it approaches zero, and you can begin to feel sure that there's really nothing happening. But in audio, there's not a whole lot you can do to make your tests more sensitive. Your measuring instruments are limited not by technology, but by the ear/brain system of very human listeners.

The dullness of our tests—their insensitivity—leaves a space where reviewers can roam free. Within that space, reviewers may wax poetic about palpability and sweetness and air and light without fear of contradiction by science. It's a space where much could be happening—apparently is happening—but where it's impossible to be sure whether it's happening or not. A lot of life is like that.

Of course, this is not the only space where audio reviewers operate. They may—and frequently do—roam outside this safe habitat, making observations about easily audible things that no doubt could, with a bit of work, be scientifically verified. But there is little incentive for audio writers to take such tests, especially when they are already sure of what they've heard.

If my argument so far seems to favor the subjectivist side, it's now time to rebalance things. On reading a paper by a young colleague, the great physicist Wolfgang Pauli (1900–1958) commented, "It's not even wrong." He meant that the ideas proposed in the paper could not even be tested. For Pauli, this was the ultimate insult.

Luckily for us, the human population is diverse. Not everyone feels as Pauli did. Yet a science-based activity without scientific constraints, in which the only distinction among tweaks that appear to be nothing more than snake-oil, well-designed amplifiers, and speakers with good dispersion characteristics are the vicissitudes of personal aural experience, makes me uncomfortable. I find myself craving some certainty, if only to put a little more space between the creations of a skilled audio designer and, say, a jar of pretty rocks.



Footnote 1: Perhaps the best example is the testing of audio codecs and lossy compression schemes intended for the delivery of digital audio. While these tests almost certainly underestimate audibility of some compression artifacts, they routinely establish the audibility of others.

Footnote 2: The Demon-Haunted World: Science as a Candle in the Dark (1995).

Footnote 3: "How Conventional Statistical Analyses Can Prevent Finding Audible Differences in Listening Tests," Leventhal, presented at the October 1985 AES Convention in New York. Preprint 2275. See also his article in Stereophile. H0 is the null hypothesis, the hypothesis that the effect under test is inaudible. Leventhal is saying, in other words, that a failure to detect audibility is not evidence of inaudibility.

Share | |
Comments
WillWeber's picture
bingo!

Thanks Jim, a nice synopsis of the reality of "scientific audio". There are indeed, unsound “scientific practice” slithering among the fields of sound (pun apologies). Some is intentional to disguise the truth; some is simply a naïve use of the tools. Such contamination has poisoned objectivity; snake venom abounds.

Once we are able to measure all the relevant parameters with sufficient precision (and report honestly) the mysticism would all but vanish; audio nirvana will become more accessible and snake oil will vaporize. The problem is to identify and confirm the key measurement parameters in the reproduction chain. This is not easy, and here we will need to do the tedious tasks of scientific listening tests. Fortunately, that approach minimizes the number of such tests, foregoing the need to “listen test” each piece of gear individually, in lieu of measurement. Someday… I am optimistic, yes.

WillW

geoffkait's picture
Knowledge is Power

Nordost's slogan "Knowledge is Power" is flashing just above JA's article, "Audio, Meet Science," at the top of the page. Ironic, huh?

Geoff Kait
Machina Dynamica
We Do Artificial Atoms Right

300Binary's picture
Call me unreliable

Uh, if I like what I hear or think I hear, it is not important which is the source. We are allowed to fantasize about recorded music, there are no musicians in my earbuds. Assuming only serious comparisons have any merit is silly. Some people seem to think others are getting ripped off if they buy a system that is not as good as they think it is. I like to decide for myself, not listen to some well reasoned argument about my little fantasy world. There is more to life than test procedure races. For some of us, anyway ;)

WillWeber's picture
unreliable but preferential

Yes 300binary, you are correct, we all have our preferences, and listening is important. That was not the point of Jim's article. Think of it this way: you should be able to buy your red car, or whatever you prefer, but with your eyes open, and without those colored glasses that are always changing tint.

WillW

300Binary's picture
preferred, at the moment

Colorless glasses are no more possible than perfect audio toys. We each have to pick the least offensive to our own mind, at this moment. It may change, or not. We are not creating maps of reality, just maps of our minds. Vanity claims my map is larger than my petty opinion... until it changes.

WillWeber's picture
colorless glasses

Actually yes, there are such things as colorless glasses, this has been quite well established, and is easily measured. The same is possible in audio, some day, as long as the designers are scientific about the evolution. That's largely how the sound quality has gotten better over the years.

But our preferences do change, as you say. Listeners are subjective. But so what? We can still better guide our selection if we enter objective metrics. That is the part that audio researchers and manufacturers are responsible for.

300Binary's picture
Easy measurements, hard truths

People do not all have identical auditory experiences.

Even the same person may enjoy a system one night and shut it off, on another.

Finding the average is not truth, it is a marketing ploy.

Lies, damn lies and statistics.

A system that pleases the most people in rigorous scientific testing is nice. It does not mean it will please everyone who hears it. And, possibly, no one that actually buys it.

You want the truth. The truth is not a naked singularity. Even audio is not that simple. Some people may be, but, that is another issue, entirely.

WillWeber's picture
truth?

Dear 300binary,

to quote:
"People do not all have identical auditory experiences.
Even the same person may enjoy a system one night and shut it off, on another."

I don't disagree about that, what fool would. And no, the truth is not simple, but if we stop searching for "it", we will be subject to swindlers. People once believed that snake oil could cure all kinds of ailments, and they felt better as a result of coughing up their nickel.

The truth is a live performance. That's what I'd like to hear from my system. I know I won't get there, but I keep trying. Reproduction of that performance can be made more accurate with more neutral equipment in the chain. Better equipment can be quantified by measurements, instrumentally. Then, the listening room is our own "component" to adjust, but I want to start with accurate equipment in the rest of the chain. If there are speakers that are more suited to my room acoustics, I want to be able to find the candidates more easily. Measurements will help here too. Then I do indeed listen and adjust the room.

I like to sing in the shower. That is an audio "tweak" that makes me sound better to me. It is not a subtle difference. It distorts the truth, but still I prefer it over my naked voice. I suspect some audio tweaks are "sweeteners" too, but might be a distortion that some listeners like. And I think there are tweaks and components that are more truthful too, by reducing distortions. Which do I spend my $$ on?

PS: If anyone wants to put their audio system in the shower, their choice.

cheers,
WillW

300Binary's picture
oh, charlatans...

Hi Will,

Science and testing are not useless, but, they cannot pick out your system for you.

It is obvious to anyone who has an Audio Precision to play with that we cannot measure everything we can hear. Just because no lab can document what some doo-dah does does not mean we can string the blackguard up.

What harm is there in selling me an expensive doo-dah that only I can hear the benefit(s) from? Is it really a moral outrage that I allow myself to be fooled by some doo-dah, but, OK that I pretend to hear musicians playing in my earpods? Do you think we must be protected from having politically incorrect aural illusions?

You want cut and dried lines between true effects and false effects. There is no such division. I may be completely blown away by a new pair or quad of tubes, but, the system may not show any measureable change.

If sparrow feathers under your blutack do it for you, great. If you can sell sparrow feathers to others, great. If putting loose change on the front top edge of your speaker lights a new fire, what, that is an immoral use of government currency?

The reason science is not the master of men is not heresy, it is it simply that science has not solved all the puzzles, yet.

There is a very competent speaker designer who sells speakers that are both technically and humanly effective. He knows the numbers and the theory and does the application very well. He also believes all amplifiers sound the same. He is brilliant, just, deaf. I am a bit more interested in things like balanced idle current in the output stage, but, I do not claim Lowthers and my earpods sound the same. Neither of us must die nor vanquish the other. We have some common ground, but, we cannot all be clones of some real true facts. It is not that simple. It can be, for one. For six billion, it is just another fantasy.

If you want to be a tweak bigot, it is a free country. But, do not claim some machine makes you narrow minded ;)

Happy Ears!

Al Marcy

WillWeber's picture
and pseudo-science

Hi Al,

Thanks for the lively discussion!

As you say: "The reason science is not the master of men is not heresy, it is it simply that science has not solved all the puzzles, yet."

That is my point really. We will get there by doing good and honest science, and we have progressed much already (although there is way too much misuse of supposed "science" currently). And I want to believe that manufacturers that practice such "religion" will benefit through higher acceptance by more satisfied customers. Purely "by the numbers" does not work now because we need more research on which parameters are important and how to measure these, a formidable task indeed. And we should never blindly just trust numbers; they should lead us to better equipment to audition and provide a reasonable expectation. We aren't measuring all the relevant parameters yet, and some not precisely enough. Amazing how sensitive the auditory system is! And complicated.

I don't disagree with you at all about preferences, they belong to each of us. My preference is accurate reproduction, which remains an illusion after all. Until we can regenerate the entire sonic wavefront, reproduction cannot be complete. The illusion tends to be better for me when using "accurate" components in the chain. If ever I did prefer some sweetening enhancements, that's fine, but I'd want to know honestly what is being modified and how to optimize the effect at the lowest cost without trying out dozens of products.

Also, I actually do use a number of tweaks, have been disappointed by a few with their pseudo-scientific claims of "magical powers"; but then some actually do work well for me. Those tend to be of the subtractive kind, removing some vibration or noise, for example.

Enjoy the music, and make some too!

Will

Jorgensen's picture
Evidence of absence

While it is true that one cannot prove non-existence, as well as that 'absence of evidence is not evidence of absence', we can reduce the likelihood that an effect exists through scientific testing. In other words, you can have such a thing as evidence against an effect. We can use randomised, double blind trials to obtain such evidence, in the face of human inclinations to subjective assessments. This is how we can be fairly sure that an intervention such as homeopathy is rubbish. Which is very handy, as it prevents public spending on useless interventions. I think blinded trials in hi fi testing is likely to yield surprising results and that the only thing preventing its use is lack of understanding of its critical importance, and fear of what it might reveal.

geoffkait's picture
The fear of what double blind testing might reveal

Homeopathy is a term employed by self styled skeptics. Have you performed scientific tests to reach this conclusion?

You say suspect that double blind tests will reveal surprising results, but isn't that a bit presumptuous, idle speculation? Let the tests reveal what they will. That's the scientific method. I'm afraid you're putting the cart before the horse.

I also suspect there is fear what blind tests might reveal with respect to controversial tweaks, the fear they might pass the test. One wonders why Skeptics of controversial tweaks never subject the tweaks they deride to scientific testing? Hmmmm....

Tootles,

Geoff Kait
Machina Dynamica

corneliuswheldon's picture
very good reserch

very nice made research made in audio technology.actually audio technology is improve in lots of field but still need to do the research in the compression.we can reduce the likelihood that an effect exists through scientific testing. In other words, you can have such a thing as evidence against an effect.
Lancaster PA web design

geoffkait's picture
"Reduce the likelihood of the effect through scientific testing"

Quote:
very nice made research made in audio technology.actually audio technology is improve in lots of field but still need to do the research in the compression.we can reduce the likelihood that an effect exists through scientific testing. In other words, you can have such a thing as evidence against an effect.

I think that's quite a nice sentiment, that an effect can be proven to not exist scientifically. Unfortunately for that sentiment, noone ever gets down to brass tacks and performs the "scientific testing" that might prove that the effect exists or not. So far, the only thing I've seen is a lot of whooping and hollering. I don't happen to think whooping and hollering is scientific evidence.

Geoff Kait
Machina Dynamica

brindoporeso's picture
burden of proof

geoffkait wrote:

“One wonders why Skeptics of controversial tweaks never subject the tweaks they deride to scientific testing?”
Hopefully, derision aside, it is because they are smart enough to realize that they are under no such obligation because that is not where the burden of proof lies. Absence of evidence is not evidence of absence,but it does not need to be. Such evidence is logically unnecessary, just as there is no need to spend millions of dollars proving that snake oil does not cure cancer. Until grounds exits for rejecting the null hypothesis, such issues as pharmacological efficacy or cold fusion simply do not arise, Failure to distinguish an onus probandi from a hole in the ground is the argumentum ad ignorantiam fallacy, which may provide merchants with an opportunity for commercial exploitation, but which, to logicians, is good only for a laugh.
It has been pointed out that certain double-blind audio tests were flawed compared to properly designed, placebo-controlled drug test. This criticism is valid, but subjectivists then routinely fail to appreciate the consequences of such failures. The goal of drug testing is to find a difference in performance between the candidate drug and a placebo. If the test is flawed such that it gives no valid results, then no opportunity has arisen to reject the null hypothesis, which therefore stands. That a tweak does not improve things requires no proof because it constitutes the null hypothesis, invalid tests of which leave it standing until a valid test can be performed. In the meantime, another cardinal principle of epistemology should not be forgotten. Even if certain subtle audio phenomena exist and even if certain audiophiles can hear them, talk remains cheap.

geoffkait's picture
You know what they say when you assume something?

"One wonders why Skeptics of controversial tweaks never subject the tweaks they deride to scientific testing?”

"Hopefully, derision aside, it is because they are smart enough to realize that they are under no such obligation because that is not where the burden of proof lies. Absence of evidence is not evidence of absence,but it does not need to be. Such evidence is logically unnecessary, just as there is no need to spend millions of dollars proving that snake oil does not cure cancer. Until grounds exits for rejecting the null hypothesis, such issues as pharmacological efficacy or cold fusion simply do not arise, Failure to distinguish an onus probandi from a hole in the ground is the argumentum ad ignorantiam fallacy, which may provide merchants with an opportunity for commercial exploitation, but which, to logicians, is good only for a laugh.

It has been pointed out that certain double-blind audio tests were flawed compared to properly designed, placebo-controlled drug test. This criticism is valid, but subjectivists then routinely fail to appreciate the consequences of such failures. The goal of drug testing is to find a difference in performance between the candidate drug and a placebo. If the test is flawed such that it gives no valid results, then no opportunity has arisen to reject the null hypothesis, which therefore stands. That a tweak does not improve things requires no proof because it constitutes the null hypothesis, invalid tests of which leave it standing until a valid test can be performed. In the meantime, another cardinal principle of epistemology should not be forgotten. Even if certain subtle audio phenomena exist and even if certain audiophiles can hear them, talk remains cheap."

You make several assumptions that are quite typical of the skeptical stance, yet quite incorrect. One is that the burden of proof falls on the manufacturer. There is, in fact, no requirement for a manaufacturer to prove either his claim for what the device does or to prove any explanation he might provide for how the device works. In fact, its perfectly OK for the manufacturer NOT to provide any explanation whatsoever. In the real world, in industry and government, as opposed to the fake, make-believe world of the died in the wool Skeptic, proof of concept must be performed by an independent third party. Independent verification and validation. Tests can be rigged and data can be falsified, so what self respecting Skeptic would accept the manufacturer's word for it, anyway? Duh!

Another false assumption you make is that the effects of controversial tweaks are always subtle. Skeptics have been using that argument for years - hoping, one assumes, to trivialize controversial tweaks, they are simply too small or too outrageous, audiophiles can only trust tweaks their science allows. LOL! Of course, if the effects were really as subtle as Skeptics indicate (not that they've actually listened to them, heaven forbid) there wouldn't be any controversy! LOL!! Skeptics would have us believe these preposterous sounding, highly dubious tweaks could have no more influence on the sound than a speck of dust. Or an artifical atom. LOL

Another false assumption you make is that there's any relationship between medical testing and audio testing. As if the Placebo Effect explains away all positive results. That's SO funny!! The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox, along the lines of controlled blind testing.

So, if Skeptics are not going to investigate the claims of controversial tweaks like good little scientists and just going to sit around in their soft loungers, then guess what? - talk really is cheap. Blah, blah and more blah.

There is nothing more humorous than a Skeptic that can't shoot straight.

http://members.cox.net/donknotts/knotts-shakiest.jpg

Geoff Kait
Machina Dynamica
We Do Artifical Atoms Right

brindoporeso's picture
phantom assumptions

Geoff Kait writes:

“You make several assumptions that are quite typical of the skeptical stance, yet quite incorrect.”

I make no assumptions. I state logic, which I did not invent, so don’t blame me.

“One is that the burden of proof falls on the manufacturer.”

It falls on a proposition. Independence of testing is part of proper experimental design, and I never said otherwise. I also don’t employ “skepticism” in the superficial colloquial sense of something one has until convinced about something. In the basic philosophical sense, it is simple knowing where the burden of proof lies. One loses it only when sanity departs. I said nothing about manufacturers. Conservatives often say, “I don’t believe in global warming BECAUSE it is touted by Al Gore, and why should I trust HIM?” You are not to trust him or anyone else. You’re supposed to read thermometers. If you want to know about nature, ask nature, not Aristotle.
“Another false assumption you make is that the effects of controversial tweaks are always subtle.”
Guess again. I referred to a hypothetical particular one for purposes of illustration, not all possible ones.
“hoping, one assumes, to trivialize controversial tweaks”
I hope for nothing, except that criticism be aimed at what I actually wrote, not what one wishes I had. The record speaks for itself. As with these “assumptions” of which you write, even if you are right about their incorrectness, tell it to someone who actually made them. As to the hearing abilities of audiophiles, it can’t be any worse then their reading comprehension, if this is any example.

“As if the Placebo Effect explains away all positive results.”
It is in fact the standard against which positive results are measured.
“The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox”
Were that the case, It would be called the placebo fallacy. Trouble is, it really happens. It is not unusual to see 60 percent of patients improve when treated with a drug (”positive effect”), 5 percent to improve when not treated and 20 to improve when treated with a placebo.
“So, if Skeptics are not going to investigate the claims of controversial tweaks like good little scientists and just going to sit around in their soft loungers, then guess what? - talk really is cheap. Blah, blah and more blah.”
Yes, because skeptics aren’t the ones doing the talking. The null hypothesis requires no Devil’s Advocate. I one presents no evidence of guilt against a criminal defendant like a good little prosecutor, then guess what? He sits the innocent. When tests, such as audio gear, are not controlled, then the cause of a reviewer’s pleasure or displeasure in the experience cannot be confidently assigned to the hardware. When tests are done “in context,” they include factors that do not effect me, such as the reviewer’s room and chair and wallpaper and family and pets, etc.

“There is nothing more humorous than a Skeptic that can't shoot straight.”
Except perhaps someone who doesn’t appreciate skepticism in the proper sense and who shoots straight at targets that do not exist. A later comment about “!12 Angry Men” is telling. It is saying that jurors could not prove the defendant’s INNOCENCE, which they needn’t. It’s there already, just like the null hypothesis.

geoffkait's picture
The placebo effect is nothing more than some contrived tool

Quote:
“As if the Placebo Effect explains away all positive results.”

It is in fact the standard against which positive results are measured.

“The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox”

Were that the case, It would be called the placebo fallacy. Trouble is, it really happens. It is not unusual to see 60 percent of patients improve when treated with a drug (”positive effect”), 5 percent to improve when not treated and 20 to improve when treated with a placebo.

The problem with your argument is that noone is saying the placebo effect never happens. Simply because the placebo effect might occur in some cases, for some listeners, logically it cannot explain away all positive results. Which is why Skeptics bring up placebo effect in the first place - to dismiss all positive results. Some positive results can be dismissed for other reasons, such as expectation bias, plain bad hearing, etc., but in the end there will be a significant number of positive results that cannot be ignored or explained away. That is precisely why the tweaks we're talking about are, in fact, controversial!

One should also consider that some audiophiles - let's call them advanced audiophiles - are more capable, through experience, of eliminating placebo type effects from their tests. The placebo effect should be lumped together with the false logic of, "believers in controversial tweaks must also believe in homeopathy and flying saucers."

Geoff Kait
Machina Dynamica

brindoporeso's picture
Fewer Errors

“The problem with your argument is that noone is saying the placebo effect never happens.”

Except perhaps you when you wrote, “The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox, along the lines of controlled blind testing.”

As you now admit, it IS more. It is a fact.

“Simply because the placebo effect might occur in some cases, for some listeners, logically it cannot explain away all positive results.”

Obviously. It defines their magnitude.

“Which is why Skeptics bring up placebo effect in the first place - to dismiss all positive results.”

Not the ones who do it right. In the drug scenario, it establishes the of the effect for which the drug is not responsible. If 20 percent of patients improve with a placebo and 60 with a drug, the drug deserves credit only for the 40 percent differential. If no skeptic ever brought up the placebo effect, no drug tests would ever be accepted for publication in scholarly journals.

“Some positive results can be dismissed for other reasons, such as expectation bias, plain bad hearing, etc., but in the end there will be a significant number of positive results that cannot be ignored or explained away. That is precisely why the tweaks we're talking about are, in fact, controversial!”

That should be precisely they they are not. Given proper experimental design, there’s the effect. Deal with it.

“One should also consider that some audiophiles - let's call them advanced audiophiles - are more capable, through experience, of eliminating placebo type effects from their tests.”

I’d just call them sane.

“The placebo effect should be lumped together with the false logic of, "believers in controversial tweaks must also believe in homeopathy and flying saucers."

Except that you just got through acknowledging that it happens, which it does.

geoffkait's picture
"Given proper experimental design..."

Fewer Errors
Submitted by brindoporeso on March 12, 2011 - 1:35am.

Quote:
“The problem with your argument is that noone is saying the placebo effect never happens.”

Except perhaps you when you wrote, “The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox, along the lines of controlled blind testing.”

As you now admit, it IS more. It is a fact.

Duh! I think you must not have your thinking cap on as my point is the placebo effect, while it might affect some listeners, can be minimized or eliminated by careful testing. You know, eliminate - as in it doesn't exist! Placebo effect, like expectation bias, is simply another variable to be controlled. Like background noise. Thus, placebo effect is, in fact, nothing more than a distraction, a contrivance dreamed up by anti-tweak Skeptics. Furthermore, if we couldn't control such things, we would never be able to progress in this hobby since all audio components, speakers, cables and isolation devices, room treatments, etc. would suddenly fall prey to the placebo effect.

Quote:
“Simply because the placebo effect might occur in some cases, for some listeners, logically it cannot explain away all positive results.”

Obviously. It defines their magnitude.

It doesn't define anything. Well, maybe in the mind of the die hard Skeptic who won't take no for an answer. Once you eliminate all the listeners who are easily fooled, and the ones who don't know what they're listening to, the ones who get hoodwinked by the placebo effect, and those who have uncontrolled expectation bias, those who are hallucinating, then you have a set of valid positive tests. End of story.
In other words, it's OK to have a hundred negative results. As long as you have at least ONE positive result by someone who can actually hear under proper test conditions, you have a problem. A big problem.

“Which is why Skeptics bring up placebo effect in the first place - to dismiss all positive results.”</p> <p>Not the ones who do it right. In the drug scenario, it establishes the of the effect for which the drug is not responsible. If 20 percent of patients improve with a placebo and 60 with a drug, the drug deserves credit only for the 40 percent differential. If no skeptic ever brought up the placebo effect, no drug tests would ever be accepted for publication in scholarly journals.[quote]</p> <p>As I've already stated, noone does proper testing on these high dubious tweaks we're talking about. In fact, noone does much proper testing of anything as far as I can tell. I don't care about your drug scenario, it's irrelevant to audio, as I already stated. I realize die hard Skeptics think it's a valid argument. I think it's a phoney argument. </p> <p>[quote wrote:
“Some positive results can be dismissed for other reasons, such as expectation bias, plain bad hearing, etc., but in the end there will be a significant number of positive results that cannot be ignored or explained away. That is precisely why the tweaks we're talking about are, in fact, controversial!”

That should be precisely they they are not. Given proper experimental design, there’s the effect. Deal with it.

When a Skeptic gets around to devising a "proper test" for the Intelligent Chip, the Clever Little Clock, the Schumann Frequency Generator, the tiny singing bowls, or the Teleportation Tweak, then we can talk turkey. Meanwhile, talk is cheap. With respect to these highly dubious tweaks, a died in the wool Skeptic wouldn't know "proper experimental design" if one came up and pissed on his leg.

Quote:
“One should also consider that some audiophiles - let's call them advanced audiophiles - are more capable, through experience, of eliminating placebo type effects from their tests.”

I’d just call them sane.

I'd call them sane, too. I knew we'd finally agree on something. So, what's with all the melodrama and angst?

Quote:
“The placebo effect should be lumped together with the false logic of, "believers in controversial tweaks must also believe in homeopathy and flying saucers."

Except that you just got through acknowledging that it happens, which it does.

Of course I acknowledged it does. Why wouldn't I? It's not as if it's some secret knowledge only known by self-proclaimed Skeptics. Everyone is free to read "Zen and the Art of Debunkery." We agree that placebo effect is a real effect and that it affects some listeners. What does that have to do with the price of tomatoes? Now, if you are trying to say only novice audiophiles are affected by the placebo effect then, Bingo, we agree again! Can I assume you are a novice, or just a standard issue tweakophobe?

Geoff Kait
Machina Dynamica
Advanced Audio Concepts

brindoporeso's picture
maybe getting somewhere

“the placebo effect, while it might affect some listeners, can be minimized or eliminated by careful testing. You know, eliminate - as in it doesn't exist!”

Even when it does occur, it is removed by subtraction, as explained above.

“It doesn't define anything. Well, maybe in the mind of the die hard Skeptic who won't take no for an answer.”

And in the mind of every student of the philosophy of science who does not deserve to flunk out and of scientists who deserve publication. It defines exactly what I said it did. Again, I didn’t write the rules. I just report them. If you are not playing by the rules of chess, then you playing a game other than chess. So it is with science.

“When a Skeptic gets around to devising a "proper test" for the Intelligent Chip, the Clever Little Clock, the Schumann Frequency Generator, the tiny singing bowls, or the Teleportation Tweak, then we can talk turkey. . . . a died in the wool Skeptic wouldn't know "proper experimental design" if one came up and pissed on his leg.”

Skeptics, in the philosophical and not the psychopathological sense, are the ONLY people who know. For simplicity, I’ll refer to these as good and bad skeptics, respectively, the latter being the ones to whom you refer. Good skeptics defer to arithmetic, while bad ones are too stupid or to cowardly to do so. Suppose I play a round of golf with Tiger Woods and he shoot 70 and I shoot 90. Imagine someone not recognizing who won and wanting to call in a referee to render an opinion. Victory is settled by arithmetic, which is not subject to opinion. Suppose I said that I really should be the winner because I am a better singer or dancer or plumber. Sorry, not relevant. So it is with scientific theories. Whatever explains the data via the fewest ad hoc hypotheses wins. This does not make it THE TRUTH, just the winner, until something better comes along. This is one of the ways wrong is avoided in science.
If it quacks like a duck, then . . . it quacks like a duck. That does not mean it is a duck, but the theory that it is a duck is a better theory that it is an oak tree. I never says it’s a duck nor that it isn’t an oak tree. I let you listen and agree that it quacks like a duck, but then roots and bark and leaves are discovered and it turns out it really is an oak tree that makes a quacking sound when the wind blows through it. Someone might say, “See? You were wrong.” I wasn’t, because I never said it was a duck and not a tree. I accurately observed that the duck theory was the best explanation of the original data. “Proving the skeptics wrong” simply cannot be done to good skeptics because they are smart to arrange thing so that being wrong is simply not an option. Where uncertainty exists, the subjunctive mood is used rather than the declarative.
The main thing is to follow Ockham” Razor, which was originally given as an imperative that essentially says, “Don’t misbehave. Don’t explain wastefully. If it ain’t broke, don’t fix it. Once you get the golf ball into the cup, stop swinging.” Alas, even some scientists have a hard time properly understanding this, as when they mistakenly think it involves probability. Fortunately, they always seem to understand it within the context of golf. I’ve never heard anyone say that the golfer with the lowest score is USUALLY the winner or PROBABLY the winner or has the best CHANCE of being the winner. So it is in science. Theories are not true or false, they are better or worse. The ideal is to evaluate them exactly as in golf. In practice, it isn’t always quite so clean cut. But more often than one might expect, it’s nothing but arithmetic, which, again, is not subject to opinion.
“Good” skepticism ultimately underlies the presumption of criminal innocence, which does not preclude conviction, which happens all the time. When conviction occurs, it in no way “proves the skeptics wrong.” “Good” skeptics are the ones who know where to begin the process. It about showing one’s work. If you are brought to trial for murder and the prosecutor says, “I got nothin’,” you walk. Even without proving your innocence? Yes, not because you didn’t do it (maybe you did), but because those bearing the burden of proof didn’t show their work. Again, this would merely be an acquittal, not a declaration that you didn’t do it, so that if it was later found that you DID do it, it would not be true that the court was “wrong.”
Bad skepticism is also related to the Duhem-Quine thesis, which holds that you weasle your way out of any result by blaming the experiment. Falsificationism is the popular, post-war philosophical stance. It is popularly thought of as saying that theories cannot be proved, only disproved. Duhem-Quine holds that you can’t confidently disprove them either.

“I'd call them sane, too. I knew we'd finally agree on something. So, what's with all the melodrama and angst?”

We probably agree on much more, but I’ve been misperceived so much it’s been a struggle. It’s as if I say, “Paris is the capitol of France,” and the response comes back: “Oh. You’re one of those France-loving socialists!” Or I get, “The trouble with your argument is . . .” when I made no argument, but stated a fact. It’s like poised response triggered any stimulus, relevant or not, as when Lucy asks Ricky for shopping money, he reluctantly agrees and she says, “Why not?” Or when Sean Hannity presented a video in which Bill Mahr said, “Not all Republicans are bigots,” and then said, “So, he thinks all Republicans are bigots.” I refer to subtle tweaks and am accused of assume that they are all subtle, an assumption that would have prevented me from using the word “subtle” because it would then be as redundant as saying “unmarried bachelors.” All I ask is that the actual text be read. I am only responsible for what I wrote, not for what someone else imagined. Where there could be “trouble” with my arguments, I simply don’t make any. I am registered as a non-partisan voter because I never studied poli sci, so I don’t feel equipped to form valid political opinions. I wait for others to visit my areas of expertise and they often obligingly allow me to enjoy the thrill of victory.

“We agree that placebo effect is a real effect and that it affects some listeners. What does that have to do with the price of tomatoes?”

You had trouble expressing that consistently, as noted above. If this settles it, cool. As to the earlier business about controls, I’m glad you realize that it is sometimes hard to get people to appreciate them. I love when people observe that 70 percent of convicts came from broken homes or smoked marijuana. I ssay, yeah, and probably 99 percent of them grew up drinking milk and wearing shoes, which is an even better correlation. So where’s your outrage against milk and shoes? That’s what controls are for: to sort these things out.

“Can I assume you are a novice, or just a standard issue tweakophobe?”

I refer you to your own comments on “assuming” posted earlier. My specific relation to audio is trivial. I was speaking abstractly about general concepts. I’m sorry for any confusion with their application to specific audio matters. I was just hoping to provide some insight from the perspective of science, which was supposed to be the subtopic of this discussion. As for logic, if one consults the following resources, even if the terminology is new to one, the principles should be intuitively obvious:

www.nizkor.org/features/fallacies
leo.stcloudstate.edu/acadwrite/logic.html
usabig.com/autonomist/fallacies.html
www.csun.edu/~dgw61315/fallacies.html
www.fallacyfiles.org

geoffkait's picture
Good Skeptic, Bad Skeptic

Quote:
And in the mind of every student of the philosophy of science who does not deserve to flunk out and of scientists who deserve publication. It defines exactly what I said it did. Again, I didn’t write the rules. I just report them. If you are not playing by the rules of chess, then you playing a game other than chess. So it is with science.

But isn't that exactly what the Bad Skeptics say, that science is on their side, and the laws of science cannot be disobeyed? "The candle of science in a world of demons," to paraphrase JA's Footnote to his article. The problem is noone can tell the Good Skeptics from the Bad Skeptics sicne they use the same arguments. Maybe we should require nametags. :-) Anything Skeptics, the Men of Science, take exception to are automatically placed in the box called Bad Science. As if all scientists think alike and agree about everything, even outrageous tweaks. That is precisely what this thread is all about. By claiming one is a Man of Science it automatically means that anything HE deems UNscientific be labeled False or a Fraud. Isn't that simply demonizing it? You might as well say it's paranormal, or pychological or delusional. You're the expert on logic, but isn't that argument an Appeal to Authority? In the specific case of JA, a PhD Theoretical Physics, anything that he's not OK with science-wise, physics-wise, must fall under the category, Hogwash. E.g., Intelligent Chip, Brilliant Pebbles, etc. Is that not the point of JA's article - that people in the industry should be more skeptical of "outrageous tweaks" - the ones that give so-called scientific types heartburn, the ones JA has been wringing his hands about for the last six years. So does that make JA a Good Skeptic or a Bad Skeptic? - you tell me, you're the expert.

Raising any scientific truism (such as the placebo effect) in the defense of science, when attacking preposterous sounding tweaks, such as the Intelligent Chip or Brilliant Pebbles is always a good strategy in the war on hard to swallow tweaks since the Laws of Physics cannot be disobeyed; if you make it appear you're solidly on the side of physics, in fact ALL science, how can you lose? LOL

And just exactly how do you propose one go about testing an audio device that purportedly has no effect on the audio signal or the acoustic wave reaching the listener's ears?

I reiterate, whether the person is a Good Skeptic or Bad Skeptic, I have yet to see ANY Skeptic actually do ANYthing except construct clever arguments. One never sees ANY of these Skeptics actually roll up his sleeves and investigate these outrageous Science-disobeying devices, the objects of so much writing. Isn't investigation really the most important part of true skepticism? I suspect most Skeptics just want to appear to be on the side that's winning, sitting comfortably in the peanut gallery.

Quote:
“When a Skeptic gets around to devising a "proper test" for the Intelligent Chip, the Clever Little Clock, the Schumann Frequency Generator, the tiny singing bowls, or the Teleportation Tweak, then we can talk turkey. . . . a died in the wool Skeptic wouldn't know "proper experimental design" if one came up and pissed on his leg.”

Skeptics, in the philosophical and not the psychopathological sense, are the ONLY people who know.

I gather you are unfamiliar with the devices I mentioned, otherwise you probably would have taken a more precautionary stance. But I can certainly understand if you are arguing from the position of a Good Skeptic, not an audiophile. What is needed is a real, Good Skeptic who understands what he's up against scientifically and audiophile-wise, an experienced open-minded listener, when it comes to these outrageous tweaks we're talking about. So far, I don't see that happening. I do, however, see people posing as Good Skeptics, perhaps also posing as audiophiles, using illogical arguments to attack any errant sympathizers of high dubious tweaks. One wonders what can their motive be - to illuminate mankind? LOL

Geoff Kait
Machina Dynamica

brindoporeso's picture
Time Out and Adios

I apologize for any confusion that may have arisen, which I will try to resolve. Commonly, the word “skepticism” refers to a form of misbehavior. The people you call skeptics are either idiots or cowards or both. Unfortunately, philosophers use similar terminology when discussing a form of behavior that is logically proper and necessary. Without it, scientists would not be doing it right and science would not be happening. So, I’ll refer to your skeptics as “idiots” and will contrast them with “scientists.” I could qualify these terms to make them more precise, but I’ll forego that in the absence of further confusion. After this, I may retreat to the philosophical realm where my terminology has been canonical since long before I showed up and causes now confusion. So. I’ll say good bye here and wish you well.

“The problem is noone can tell the Good Skeptics from the Bad Skeptics sicne they use the same arguments.”

Idiots must eventually resort to bogus ones. Editors don’t decide what gets published in the scholarly literature based on a coin toss. Standards exist, and upward-looking snobbery (the pot calling the sugar black) represents just such desperation.

“You're the expert on logic, but isn't that argument an Appeal to Authority?”

Only if I appeal to myself, which I don’t. If logic isn’t intuitively obvious to you, then you have no other option than appeal to authority and have no way to know if you’re right or wrong. It’s supposed to be about recognition, not trust. I even advised looking past Aristotle. The goal is to be smart enough to make such evaluations oneself. Of course, this can be taken to sociological extremes. When friends talk about people who make more money than I do (watch out for that argumentum ad crumenam) I say, “Yeah. Know who else made more? Lassie, but money did not earn Lassie the executive pole of the dog leash.” If I say “2+2=4” and you can’t tell if I’m right or wrong, what is there to say except, “Fetch!” You mention the need for open-minded listeners. The mind of the scientist is open, but not empty. It knows what to do with what goes in.

“And just exactly how do you propose one go about testing an audio device that purportedly has no effect on the audio signal or the acoustic wave reaching the listener's ears?”

Again, I’m talking abstractly and I have no dog in this fight. I merely quote rule books that I did not write (though I could have because I understand them). If I said “2+2=4” and you said “2+2=5,” you’d be disagreeing not with me but with arithmetic. I’m just the messenger. Not only is it futile to kill the messenger, I’ve been witnessing the killing of other messengers with other messages. No problem. Not even sticks and stones can break my bones when hurled not at me but at a straw man (or at least a different one). No harm, no foul. (The consolation was the delicious irony in your comment about MY bad aim.)

“One never sees ANY of these Skeptics actually roll up his sleeves and investigate these outrageous Science-disobeying devices, the objects of so much writing.”

See my original post. Whether talk is cheap or not, when there is none at all, some people misbehave. When selecting a jury for a criminal case, lawyers like to ask prospective jurors what verdict they would render if asked to do so before the trial was conducted. Any answer other than “not guilty” should get one disqualified because it demonstrates a lack of understanding of the system. Where there is no talk, there is acquittal. Where there is no talk, the null hypothesis persists. You expressed doubt that certain people would ever design a proper test for something. Whether talk is cheap or not, the null hypothesis is simply the observation that until SOMEBODY, ANYBODY designs and performs such a test, there is nothing about which to talk. Irresponsible impatience in such situations gets people denied publication in the scholarly literature, though this may not be such a significant issue on the recreational or commercial level.

“I suspect most Skeptics just want to appear to be on the side that's winning.”

Perhaps. The goal is to know what constitutes victory. In science, it’s often as easy as in golf, hence my earlier illustration.

billmilosz's picture
Protecting their ad revenue

Naturally, Stereophile is pointing out that you can't PROVE that NO ONE can hear A/B differences between two TOSLINK interconnects (for example) - and in terms of pure logic this is true. You can't "prove a negative" in this way.

But we don't live in a world of Platonic ideals and pure logic. We live in a PRACTICAL subset of such purity. So I say, if it has not been DEMONSTRATED IN OBJECTIVE BLIND TESTS that audible differences exist between two TOSLINK cables (for example) then I am not willing to SPEND MY MONEY on such goods! It may be logically correct that you cannot prove that NO differences exist, but until your prove in a practical way that audible differences DO exist, I'll hang on to my money, thank you very much.

Stereophile has a vested interest in protecting their subjectivist position, and so, ipso facto, they are not to be trusted.

But, if you DO buy their line, please contact me- I have a bridge for sale in Brooklyn, and if you place your speaker wire on this bridge for at least 24 hours it will increase the resolution and soundstage of your system. (Go ahead, prove that it won't!)

John Atkinson's picture
Re: Protecting their ad revenue
billmilosz wrote "Naturally, Stereophile is pointing out that you can't PROVE that NO ONE can hear A/B differences between two TOSLINK interconnects (for example) - and in terms of pure logic this is true. You can't "prove a negative" in this way."

That is correct. And certainly it is possible to measure differences between 2 Toslink cables, in terms of bandwidth, which in turn lead to different amounts of jitter, depending on what D/A processor is in use. Audibility of the effects of that jitter will also depend on the D/A in use.

"Stereophile has a vested interest in protecting their subjectivist position, and so, ipso facto, they are not to be trusted."

With respect, this seems a logical non sequitur, given that the essay that triggered your comment and that questioned the effectiveness of unlikely-sounding tweaks was indeed published in Stereophile. So by your own argument, Jim Austin's essay is itself not to be trusted. :-)

John Atkinson
Editor, Stereophile

geoffkait's picture
"We live in a PRACTICAL subset of such purity."

Quote:
But we don't live in a world of Platonic ideals and pure logic. We live in a PRACTICAL subset of such purity. So I say, if it has not been DEMONSTRATED IN OBJECTIVE BLIND TESTS that audible differences exist between two TOSLINK cables (for example) then I am not willing to SPEND MY MONEY on such goods! It may be logically correct that you cannot prove that NO differences exist, but until your prove in a practical way that audible differences DO exist, I'll hang on to my money, thank you very much.

Looks, it's really very simple. The ONLY real reason to purchase a piece of audio gear or cable or accessory is if it's better than what you already have in terms of sound quality. Now comes the difficult part. You have to obtain the piece of gear or cable or accesssory and insert it into your system. Only then can you evaluate its effect on the sound. Who, especially someone who's afraid of making a big mistake, would take someone else's word for the sound of an amp, a speaker, a cable or a tweak? You have heard of 30 day trial periods, haven't you? These demands for proof sound eerily like the lone holdout in 12 Angry Men, "But you can't PROVE it!"

Look, when you go out to purchase a TV do you conduct controlled blind tests to figure out which brand to buy? Of course you don't. That would be ridiculous, right? You pick the TV that has the best picture at your price point. That's what we all do. No data, no explanation of operation, no controlled blind tests. That would be ridiculous, right? Now, how is picking audio gear or cables or tweaks any different? Isn't it much more logical and PRACTICAL to pick audio stuff according to how it sounds rather than rely on somebody out there conducting controlled blind tests (which - for your information - noone ever does, anyway). LOL

Cheerio

Geoff Kait
Machina Dynamica

LytleSound's picture
Science, No. This is a discussion of the statistics

Acoustics is the oldest field within Physics, which, in turn, is the oldest field of the sciences. Within acoustics falls both electroacoustics and psychoacoustics. Statistics is a tool for both of these sciences; applied more for the latter than for the former. When it comes down to determining if system A sounds different from system B, the realm of psychoacoustics has been entered and an entire set of statistical issues arrive.

Issue 1: for whom must system A be different from B? If trained/experienced listeners are used, the perception of the outcome may be different. So-called golden ears are usually experienced listeners for the type of comparison being conducted. Inexperienced listeners, actually harder to define, are the "man off the street." Issue 2: to what level of statistical significance should this difference be required to meet? This is typically set at the 90% level for behavioral work, meaning that the outcome has a 90% likelihood of being repeated if the exact same comparison methods were to be repeated. Issue 3: what statistical power is required? This is an illusive concept to many, but can be distilled to the issue of the number of listeners to use in the study to determine the expected outcome at the level of significance selected and is usually based on the results of a smaller pilot study. Issue 4: will the test be blind or open? If blind, an ABX comparison may be performed, if open, only an AB comparison may be performed. Issue 5: will the test be sequential, relying upon auditory memory, or immediate, where both system A and B are present? Auditory memory is assumed to be poor for the untrained listener and reputed to be good for the golden ear.

Seldom have all of these issues been addressed for audiophile concerns. And, yet the outcomes of those comparisons that have been done "scientifically" (which really means that some level of statistical rigor was applied) are either held to by proponents or denied by opponents, almost independent of the comparison itself.

Notice that I haven't asked about the issue of what is better or worse, just if there is a difference. Also, going to Issue 1, I don't remember a comparative study of golden ears to inexperienced listeners. I have experience with golden ears in an industry where a golden ear was required to listen to every product before it was shipped. Regardless of what any other test showed, the golden ear's decision could not be overruled. The golden ear most often could point out the flaw so that the product could be fixed before being shipped - after once again getting by the same golden ear - or scrapped. None of these golden ears purported to be audiophiles; they weren't culled from a group of audiophiles nor did most even listen to "music on the hifi." And, again, their criteria weren't better versus worse or same versus different, but pass versus fail.

By the way, Acoustics is also among the least understood fields in science, particularly as compared to another branch of physics such as Optics.

geoffkait's picture
The Wide, Wide World of Acoustics

Quote:
Acoustics is the oldest field within Physics, which, in turn, is the oldest field of the sciences. Within acoustics falls both electroacoustics and psychoacoustics. Statistics is a tool for both of these sciences; applied more for the latter than for the former. When it comes down to determining if system A sounds different from system B, the realm of psychoacoustics has been entered and an entire set of statistical issues arrive.

You said, "Acoustics is the oldest filed within physics." It's interesting to compare/contrast that statement, which I'm not doubting, to your last statement, "Acoustics is also among the least understood fields in science," which I also don't doubt, but find ironic and analogous to audiophiles who begin their sentences with, "I've been in this hobby for 40 years so you can believe me when I say...."

Quote:
Issue 1: for whom must system A be different from B? If trained/experienced listeners are used, the perception of the outcome may be different. So-called golden ears are usually experienced listeners for the type of comparison being conducted. Inexperienced listeners, actually harder to define, are the "man off the street." Issue 2: to what level of statistical significance should this difference be required to meet? This is typically set at the 90% level for behavioral work, meaning that the outcome has a 90% likelihood of being repeated if the exact same comparison methods were to be repeated. Issue 3: what statistical power is required? This is an illusive concept to many, but can be distilled to the issue of the number of listeners to use in the study to determine the expected outcome at the level of significance selected and is usually based on the results of a smaller pilot study. Issue 4: will the test be blind or open? If blind, an ABX comparison may be performed, if open, only an AB comparison may be performed. Issue 5: will the test be sequential, relying upon auditory memory, or immediate, where both system A and B are present? Auditory memory is assumed to be poor for the untrained listener and reputed to be good for the golden ear.

I'm actually a big fan of auditory memory and find it rather surprising that others have trouble in this area. Oh, well. I do find that repeated testing or long listening sessions can get tiresome, however. One thing that probably should be mentioned is the Audio System used for testing. Do you have an idea how far one must go to find a system one can tolerate listening to for more than 5 minutes? - a system that is not overwrought or distorted, especially at higher volumes, or hideously thin and threadbare, with no resemblance to a high end system? Who is to be The Arbiter of Sound for the Test System - a cynical, all-thumbs tweakophobe with an ax to grind? LOL

Quote:
Seldom have all of these issues been addressed for audiophile concerns. And, yet the outcomes of those comparisons that have been done "scientifically" (which really means that some level of statistical rigor was applied) are either held to by proponents or denied by opponents, almost independent of the comparison itself.

I dunno, seems to me that statistics and data have been overdone. An obvious example is the old Japanese amplifiers of yore with great THD specs that didn't fair too well at all when it came to actually listening to them, whereas tube amps with Order of Magnitude higher THD are much more "musical" and actually sound less distorted. And how do you propose testing such things as Intelligent Chips, Cleevr Little Clocks, Schumann Frequency Generators, Mpingo Discs, crystals, hairdryers with tourmaline?

Quote:
Notice that I haven't asked about the issue of what is better or worse, just if there is a difference. Also, going to Issue 1, I don't remember a comparative study of golden ears to inexperienced listeners. I have experience with golden ears in an industry where a golden ear was required to listen to every product before it was shipped. Regardless of what any other test showed, the golden ear's decision could not be overruled. The golden ear most often could point out the flaw so that the product could be fixed before being shipped - after once again getting by the same golden ear - or scrapped. None of these golden ears purported to be audiophiles; they weren't culled from a group of audiophiles nor did most even listen to "music on the hifi." And, again, their criteria weren't better versus worse or same versus different, but pass versus fail.

By the way, Acoustics is also among the least understood fields in science, particularly as compared to another branch of physics such as Optics.

As I stated above, what's missing is the audio system that will be used for the test - who's assembling it, who decides what it contains, what room treatment, if any, is used, what isolation devices, if any, are used and who decides whether the sound achieved for the Test System is worth anything for purposes of testing?

Sincerely,

Geoff kait
Machina Dynamica
We Do Artifical Atoms Right

Jorgensen's picture
On homeophathy

Geoff - homeophathy is not most commonly a term employed by self termed sceptics, as you write. Unfortunately, it is much more commonly used by homeopathists trying to pass off their bogus philosophy to those who hope for a cure for their ailments. This should be prohibited by law, just like quackery is today.

I haven't done any research in homepathy, as you suggest I might should have done before complaining about it. However, this wouldn't be necessary, as the uselessness of the intervention has already been extensively documented, as summed up in a meta-analysis in Lancet in 2005 (Shang et al. Lancet 2005; 366(9487):726-32). I have, however, done lots of other research. I am 35 years old, but you will find 38 'hits' to my name on PubMed, including in such journals as BMJ, Lancet Oncology and the New England Journal of Medicine. I teach scientific theory at both university and post graduate level. You do not have to do research in a certain topic to form an opinion about it. You just have to read the literature, if one of reasonable quality exists.

What I aimed at in my text when I referred to surprinsing results was not whether exotic tweaks would fail in a double blinded test. I was thinking about how some people might become disappointed in their ability to hear a difference between two audio components, and therefore also regret large investiments. I agree that this was not stated clearly enough in my text, but I can also see how you would be likely to respond in the way you did, having seen your web page about the 'Brilliant Pebbles'. I encourage all sterophile readers to do the same - it is deeply fascinating: http://www.machinadynamica.com/machina31.htm.

Let me remind you that science tends to select the most plausible hypotheses to test first.

geoffkait's picture
Science selects the most plausible hypotheses to test first

Quote:

I haven't done any research in homepathy, as you suggest I might should have done before complaining about it. However, this wouldn't be necessary, as the uselessness of the intervention has already been extensively documented, as summed up in a meta-analysis in Lancet in 2005 (Shang et al. Lancet 2005; 366(9487):726-32). I have, however, done lots of other research. I am 35 years old, but you will find 38 'hits' to my name on PubMed, including in such journals as BMJ, Lancet Oncology and the New England Journal of Medicine. I teach scientific theory at both university and post graduate level. You do not have to do research in a certain topic to form an opinion about it. You just have to read the literature, if one of reasonable quality exists.

I agree that this was not stated clearly enough in my text, but I can also see how you would be likely to respond in the way you did, having seen your web page about the 'Brilliant Pebbles'. I encourage all sterophile readers to do the same - it is deeply fascinating:

http://www.machinadynamica.com/machina31.htm.

Let me remind you that science tends to select the most plausible hypotheses to test first.

....................................................

I also encourage more Stereophile readers to see my page about Brilliant Pebbles. I find the subject fascinating, too. And I often encounter folks with expertise in some area or another who express strong reactions to the idea of using crystal structures for audio applications. LOL The theoretical page for Brilliant Pebbles is at:

http://www.machinadynamica.com/machina17.htm

You are most likely correct - you do not have to do research on a certain topic to form an opinion about it. In fact, you apparently don't even have to review the available literature. That has been made abundantly clear by JA in his article (the subject of the OP) as well as innumberable times on this and many other audio forums.

Cheers,

Geoff Kait
Machina Dynamica
Advanced Audio Concepts

Site Map / Direct Links