Audio, Meet Science

For a field based on science, high-end audio has a relationship with its parent discipline that is regrettably complex. Even as they enjoy science's technological fruits, many audiophiles reject the very methods—scientific testing—that made possible audio in the home. That seems strange to me.

Of course, there are reasons—not necessarily good ones—why some people in audio have a low opinion of science. The entire hobby/industry suffered a grievous blow a generation or so ago, when too much focus was put on something—how an amplifier measured—that resembled science, and too little on how such products actually sounded. A little later, the Compact Disc's promise of technical "perfection" yielded dubious sonic benefits when compared with an older, simpler medium, the LP. And there has been, over the years, no shortage of self-identified scientific types manipulating purportedly scientific tests, or ignoring inconvenient test results, to support their preconceptions.

Subjectivists, meanwhile, sometimes seem to intentionally hold themselves up for ridicule. A few audio writers, especially for the online 'zines, seem eager to prostitute themselves for the latest preposterous product—the Intelligent Chip, the Shakti Hallograph Soundfield Optimizer, the Machina Dynamica Brilliant Pebbles, the Marigo Audio Lab Dots, the Tice Clock. Meanwhile, some prominent industry folks have consistently failed, in my opinion, to maintain a sufficiently skeptical posture toward such products. It doesn't help that in audio there has been a long tradition—a dogma, even—of "trusting your ears," despite abundant evidence from neuroscience and cognitive psychology that our ears and other senses, though supremely acute, are supremely unreliable.

Yet science—and scientific testing—has much to offer audio. Audiophiles ought to embrace scientific methods with the same healthy skepticism with which they embrace sighted, subjective equipment evaluations: as a tool that, though subject to misuse, is invaluable in the hands of honest people with the right set of skills.

One form of testing that's especially important—and especially controversial—is the use of rigorous methods to validate apparent perceptions: Did you really hear what you thought you heard? Is the sound really different with that new amplifier/cable/CD player from what it was with the old one? Does putting that photo in the freezer really change the way the system sounds?

Rigorous tests can offer scientific validation of subjective observations.(footnote 1) Such validation doesn't come easily, but when it comes it's a force for truth, justice, and the American way.

Such tests have two possible outcomes: Either you establish scientifically that there's a difference between A and B, or you fail to establish a difference. And here's a crucial point that's often overlooked by people on the objectivist side: While a positive result establishes the reality of a perception to a certain level of confidence, a null result—a failure to reliably detect a difference—does not indicate the nonexistence of that difference.

To cite an oft-quoted phrase that's sometimes attributed to Carl Sagan, "absence of evidence is not evidence of absence" (footnote 2). Or, to cite Les Leventhal in an AES paper, "Properly speaking, a statistical conclusion about H0 from a significance test cannot justify 'accepting' the scientific hypothesis that differences are inaudible" (footnote 3).

In simpler language: Differences we think we hear but that testing fails to validate may nonetheless be real. An experiment that fails to show a difference between the sounds produced by two amplifiers does not indicate that no audible difference exists.

Many of the rigorous listening tests that are relevant to audio are difficult and tiring to perform, requiring serious concentration, many repetitions, and sometimes heavy lifting. Things can be made easier by using many listeners at once, but then the only conclusions you can draw are about the average characteristics of the group. The group average may not permit distinguishing cable A from cable B, but that doesn't mean a particularly golden-eared member of the group can't.

One of the beautiful things about science is that often you can make your experiments more sensitive by applying new technologies. The likelihood of a non-null result can be squeezed and squeezed until it approaches zero, and you can begin to feel sure that there's really nothing happening. But in audio, there's not a whole lot you can do to make your tests more sensitive. Your measuring instruments are limited not by technology, but by the ear/brain system of very human listeners.

The dullness of our tests—their insensitivity—leaves a space where reviewers can roam free. Within that space, reviewers may wax poetic about palpability and sweetness and air and light without fear of contradiction by science. It's a space where much could be happening—apparently is happening—but where it's impossible to be sure whether it's happening or not. A lot of life is like that.

Of course, this is not the only space where audio reviewers operate. They may—and frequently do—roam outside this safe habitat, making observations about easily audible things that no doubt could, with a bit of work, be scientifically verified. But there is little incentive for audio writers to take such tests, especially when they are already sure of what they've heard.

If my argument so far seems to favor the subjectivist side, it's now time to rebalance things. On reading a paper by a young colleague, the great physicist Wolfgang Pauli (1900–1958) commented, "It's not even wrong." He meant that the ideas proposed in the paper could not even be tested. For Pauli, this was the ultimate insult.

Luckily for us, the human population is diverse. Not everyone feels as Pauli did. Yet a science-based activity without scientific constraints, in which the only distinction among tweaks that appear to be nothing more than snake-oil, well-designed amplifiers, and speakers with good dispersion characteristics are the vicissitudes of personal aural experience, makes me uncomfortable. I find myself craving some certainty, if only to put a little more space between the creations of a skilled audio designer and, say, a jar of pretty rocks.



Footnote 1: Perhaps the best example is the testing of audio codecs and lossy compression schemes intended for the delivery of digital audio. While these tests almost certainly underestimate audibility of some compression artifacts, they routinely establish the audibility of others.

Footnote 2: The Demon-Haunted World: Science as a Candle in the Dark (1995).

Footnote 3: "How Conventional Statistical Analyses Can Prevent Finding Audible Differences in Listening Tests," Leventhal, presented at the October 1985 AES Convention in New York. Preprint 2275. See also his article in Stereophile. H0 is the null hypothesis, the hypothesis that the effect under test is inaudible. Leventhal is saying, in other words, that a failure to detect audibility is not evidence of inaudibility.

COMMENTS
WillWeber's picture

Thanks Jim, a nice synopsis of the reality of "scientific audio". There are indeed, unsound “scientific practice” slithering among the fields of sound (pun apologies). Some is intentional to disguise the truth; some is simply a naïve use of the tools. Such contamination has poisoned objectivity; snake venom abounds.

Once we are able to measure all the relevant parameters with sufficient precision (and report honestly) the mysticism would all but vanish; audio nirvana will become more accessible and snake oil will vaporize. The problem is to identify and confirm the key measurement parameters in the reproduction chain. This is not easy, and here we will need to do the tedious tasks of scientific listening tests. Fortunately, that approach minimizes the number of such tests, foregoing the need to “listen test” each piece of gear individually, in lieu of measurement. Someday… I am optimistic, yes.

WillW

geoffkait's picture

Nordost's slogan "Knowledge is Power" is flashing just above JA's article, "Audio, Meet Science," at the top of the page. Ironic, huh?

Geoff Kait
Machina Dynamica
We Do Artificial Atoms Right

300Binary's picture

Uh, if I like what I hear or think I hear, it is not important which is the source. We are allowed to fantasize about recorded music, there are no musicians in my earbuds. Assuming only serious comparisons have any merit is silly. Some people seem to think others are getting ripped off if they buy a system that is not as good as they think it is. I like to decide for myself, not listen to some well reasoned argument about my little fantasy world. There is more to life than test procedure races. For some of us, anyway ;)

WillWeber's picture

Yes 300binary, you are correct, we all have our preferences, and listening is important. That was not the point of Jim's article. Think of it this way: you should be able to buy your red car, or whatever you prefer, but with your eyes open, and without those colored glasses that are always changing tint.

WillW

300Binary's picture

Colorless glasses are no more possible than perfect audio toys. We each have to pick the least offensive to our own mind, at this moment. It may change, or not. We are not creating maps of reality, just maps of our minds. Vanity claims my map is larger than my petty opinion... until it changes.

WillWeber's picture

Actually yes, there are such things as colorless glasses, this has been quite well established, and is easily measured. The same is possible in audio, some day, as long as the designers are scientific about the evolution. That's largely how the sound quality has gotten better over the years.

But our preferences do change, as you say. Listeners are subjective. But so what? We can still better guide our selection if we enter objective metrics. That is the part that audio researchers and manufacturers are responsible for.

300Binary's picture

People do not all have identical auditory experiences.

Even the same person may enjoy a system one night and shut it off, on another.

Finding the average is not truth, it is a marketing ploy.

Lies, damn lies and statistics.

A system that pleases the most people in rigorous scientific testing is nice. It does not mean it will please everyone who hears it. And, possibly, no one that actually buys it.

You want the truth. The truth is not a naked singularity. Even audio is not that simple. Some people may be, but, that is another issue, entirely.

WillWeber's picture

Dear 300binary,

to quote:
"People do not all have identical auditory experiences.
Even the same person may enjoy a system one night and shut it off, on another."

I don't disagree about that, what fool would. And no, the truth is not simple, but if we stop searching for "it", we will be subject to swindlers. People once believed that snake oil could cure all kinds of ailments, and they felt better as a result of coughing up their nickel.

The truth is a live performance. That's what I'd like to hear from my system. I know I won't get there, but I keep trying. Reproduction of that performance can be made more accurate with more neutral equipment in the chain. Better equipment can be quantified by measurements, instrumentally. Then, the listening room is our own "component" to adjust, but I want to start with accurate equipment in the rest of the chain. If there are speakers that are more suited to my room acoustics, I want to be able to find the candidates more easily. Measurements will help here too. Then I do indeed listen and adjust the room.

I like to sing in the shower. That is an audio "tweak" that makes me sound better to me. It is not a subtle difference. It distorts the truth, but still I prefer it over my naked voice. I suspect some audio tweaks are "sweeteners" too, but might be a distortion that some listeners like. And I think there are tweaks and components that are more truthful too, by reducing distortions. Which do I spend my $$ on?

PS: If anyone wants to put their audio system in the shower, their choice.

cheers,
WillW

300Binary's picture

Hi Will,

Science and testing are not useless, but, they cannot pick out your system for you.

It is obvious to anyone who has an Audio Precision to play with that we cannot measure everything we can hear. Just because no lab can document what some doo-dah does does not mean we can string the blackguard up.

What harm is there in selling me an expensive doo-dah that only I can hear the benefit(s) from? Is it really a moral outrage that I allow myself to be fooled by some doo-dah, but, OK that I pretend to hear musicians playing in my earpods? Do you think we must be protected from having politically incorrect aural illusions?

You want cut and dried lines between true effects and false effects. There is no such division. I may be completely blown away by a new pair or quad of tubes, but, the system may not show any measureable change.

If sparrow feathers under your blutack do it for you, great. If you can sell sparrow feathers to others, great. If putting loose change on the front top edge of your speaker lights a new fire, what, that is an immoral use of government currency?

The reason science is not the master of men is not heresy, it is it simply that science has not solved all the puzzles, yet.

There is a very competent speaker designer who sells speakers that are both technically and humanly effective. He knows the numbers and the theory and does the application very well. He also believes all amplifiers sound the same. He is brilliant, just, deaf. I am a bit more interested in things like balanced idle current in the output stage, but, I do not claim Lowthers and my earpods sound the same. Neither of us must die nor vanquish the other. We have some common ground, but, we cannot all be clones of some real true facts. It is not that simple. It can be, for one. For six billion, it is just another fantasy.

If you want to be a tweak bigot, it is a free country. But, do not claim some machine makes you narrow minded ;)

Happy Ears!

Al Marcy

WillWeber's picture

Hi Al,

Thanks for the lively discussion!

As you say: "The reason science is not the master of men is not heresy, it is it simply that science has not solved all the puzzles, yet."

That is my point really. We will get there by doing good and honest science, and we have progressed much already (although there is way too much misuse of supposed "science" currently). And I want to believe that manufacturers that practice such "religion" will benefit through higher acceptance by more satisfied customers. Purely "by the numbers" does not work now because we need more research on which parameters are important and how to measure these, a formidable task indeed. And we should never blindly just trust numbers; they should lead us to better equipment to audition and provide a reasonable expectation. We aren't measuring all the relevant parameters yet, and some not precisely enough. Amazing how sensitive the auditory system is! And complicated.

I don't disagree with you at all about preferences, they belong to each of us. My preference is accurate reproduction, which remains an illusion after all. Until we can regenerate the entire sonic wavefront, reproduction cannot be complete. The illusion tends to be better for me when using "accurate" components in the chain. If ever I did prefer some sweetening enhancements, that's fine, but I'd want to know honestly what is being modified and how to optimize the effect at the lowest cost without trying out dozens of products.

Also, I actually do use a number of tweaks, have been disappointed by a few with their pseudo-scientific claims of "magical powers"; but then some actually do work well for me. Those tend to be of the subtractive kind, removing some vibration or noise, for example.

Enjoy the music, and make some too!

Will

Jorgensen's picture

While it is true that one cannot prove non-existence, as well as that 'absence of evidence is not evidence of absence', we can reduce the likelihood that an effect exists through scientific testing. In other words, you can have such a thing as evidence against an effect. We can use randomised, double blind trials to obtain such evidence, in the face of human inclinations to subjective assessments. This is how we can be fairly sure that an intervention such as homeopathy is rubbish. Which is very handy, as it prevents public spending on useless interventions. I think blinded trials in hi fi testing is likely to yield surprising results and that the only thing preventing its use is lack of understanding of its critical importance, and fear of what it might reveal.

geoffkait's picture

Homeopathy is a term employed by self styled skeptics. Have you performed scientific tests to reach this conclusion?

You say suspect that double blind tests will reveal surprising results, but isn't that a bit presumptuous, idle speculation? Let the tests reveal what they will. That's the scientific method. I'm afraid you're putting the cart before the horse.

I also suspect there is fear what blind tests might reveal with respect to controversial tweaks, the fear they might pass the test. One wonders why Skeptics of controversial tweaks never subject the tweaks they deride to scientific testing? Hmmmm....

Tootles,

Geoff Kait
Machina Dynamica

corneliuswheldon's picture

very nice made research made in audio technology.actually audio technology is improve in lots of field but still need to do the research in the compression.we can reduce the likelihood that an effect exists through scientific testing. In other words, you can have such a thing as evidence against an effect.
Lancaster PA web design

geoffkait's picture
Quote:

very nice made research made in audio technology.actually audio technology is improve in lots of field but still need to do the research in the compression.we can reduce the likelihood that an effect exists through scientific testing. In other words, you can have such a thing as evidence against an effect.

I think that's quite a nice sentiment, that an effect can be proven to not exist scientifically. Unfortunately for that sentiment, noone ever gets down to brass tacks and performs the "scientific testing" that might prove that the effect exists or not. So far, the only thing I've seen is a lot of whooping and hollering. I don't happen to think whooping and hollering is scientific evidence.

Geoff Kait
Machina Dynamica

brindoporeso's picture

geoffkait wrote:

“One wonders why Skeptics of controversial tweaks never subject the tweaks they deride to scientific testing?”
Hopefully, derision aside, it is because they are smart enough to realize that they are under no such obligation because that is not where the burden of proof lies. Absence of evidence is not evidence of absence,but it does not need to be. Such evidence is logically unnecessary, just as there is no need to spend millions of dollars proving that snake oil does not cure cancer. Until grounds exits for rejecting the null hypothesis, such issues as pharmacological efficacy or cold fusion simply do not arise, Failure to distinguish an onus probandi from a hole in the ground is the argumentum ad ignorantiam fallacy, which may provide merchants with an opportunity for commercial exploitation, but which, to logicians, is good only for a laugh.
It has been pointed out that certain double-blind audio tests were flawed compared to properly designed, placebo-controlled drug test. This criticism is valid, but subjectivists then routinely fail to appreciate the consequences of such failures. The goal of drug testing is to find a difference in performance between the candidate drug and a placebo. If the test is flawed such that it gives no valid results, then no opportunity has arisen to reject the null hypothesis, which therefore stands. That a tweak does not improve things requires no proof because it constitutes the null hypothesis, invalid tests of which leave it standing until a valid test can be performed. In the meantime, another cardinal principle of epistemology should not be forgotten. Even if certain subtle audio phenomena exist and even if certain audiophiles can hear them, talk remains cheap.

geoffkait's picture

"One wonders why Skeptics of controversial tweaks never subject the tweaks they deride to scientific testing?”

"Hopefully, derision aside, it is because they are smart enough to realize that they are under no such obligation because that is not where the burden of proof lies. Absence of evidence is not evidence of absence,but it does not need to be. Such evidence is logically unnecessary, just as there is no need to spend millions of dollars proving that snake oil does not cure cancer. Until grounds exits for rejecting the null hypothesis, such issues as pharmacological efficacy or cold fusion simply do not arise, Failure to distinguish an onus probandi from a hole in the ground is the argumentum ad ignorantiam fallacy, which may provide merchants with an opportunity for commercial exploitation, but which, to logicians, is good only for a laugh.

It has been pointed out that certain double-blind audio tests were flawed compared to properly designed, placebo-controlled drug test. This criticism is valid, but subjectivists then routinely fail to appreciate the consequences of such failures. The goal of drug testing is to find a difference in performance between the candidate drug and a placebo. If the test is flawed such that it gives no valid results, then no opportunity has arisen to reject the null hypothesis, which therefore stands. That a tweak does not improve things requires no proof because it constitutes the null hypothesis, invalid tests of which leave it standing until a valid test can be performed. In the meantime, another cardinal principle of epistemology should not be forgotten. Even if certain subtle audio phenomena exist and even if certain audiophiles can hear them, talk remains cheap."

You make several assumptions that are quite typical of the skeptical stance, yet quite incorrect. One is that the burden of proof falls on the manufacturer. There is, in fact, no requirement for a manaufacturer to prove either his claim for what the device does or to prove any explanation he might provide for how the device works. In fact, its perfectly OK for the manufacturer NOT to provide any explanation whatsoever. In the real world, in industry and government, as opposed to the fake, make-believe world of the died in the wool Skeptic, proof of concept must be performed by an independent third party. Independent verification and validation. Tests can be rigged and data can be falsified, so what self respecting Skeptic would accept the manufacturer's word for it, anyway? Duh!

Another false assumption you make is that the effects of controversial tweaks are always subtle. Skeptics have been using that argument for years - hoping, one assumes, to trivialize controversial tweaks, they are simply too small or too outrageous, audiophiles can only trust tweaks their science allows. LOL! Of course, if the effects were really as subtle as Skeptics indicate (not that they've actually listened to them, heaven forbid) there wouldn't be any controversy! LOL!! Skeptics would have us believe these preposterous sounding, highly dubious tweaks could have no more influence on the sound than a speck of dust. Or an artifical atom. LOL

Another false assumption you make is that there's any relationship between medical testing and audio testing. As if the Placebo Effect explains away all positive results. That's SO funny!! The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox, along the lines of controlled blind testing.

So, if Skeptics are not going to investigate the claims of controversial tweaks like good little scientists and just going to sit around in their soft loungers, then guess what? - talk really is cheap. Blah, blah and more blah.

There is nothing more humorous than a Skeptic that can't shoot straight.

http://members.cox.net/donknotts/knotts-shakiest.jpg

Geoff Kait
Machina Dynamica
We Do Artifical Atoms Right

brindoporeso's picture

Geoff Kait writes:

“You make several assumptions that are quite typical of the skeptical stance, yet quite incorrect.”

I make no assumptions. I state logic, which I did not invent, so don’t blame me.

“One is that the burden of proof falls on the manufacturer.”

It falls on a proposition. Independence of testing is part of proper experimental design, and I never said otherwise. I also don’t employ “skepticism” in the superficial colloquial sense of something one has until convinced about something. In the basic philosophical sense, it is simple knowing where the burden of proof lies. One loses it only when sanity departs. I said nothing about manufacturers. Conservatives often say, “I don’t believe in global warming BECAUSE it is touted by Al Gore, and why should I trust HIM?” You are not to trust him or anyone else. You’re supposed to read thermometers. If you want to know about nature, ask nature, not Aristotle.
“Another false assumption you make is that the effects of controversial tweaks are always subtle.”
Guess again. I referred to a hypothetical particular one for purposes of illustration, not all possible ones.
“hoping, one assumes, to trivialize controversial tweaks”
I hope for nothing, except that criticism be aimed at what I actually wrote, not what one wishes I had. The record speaks for itself. As with these “assumptions” of which you write, even if you are right about their incorrectness, tell it to someone who actually made them. As to the hearing abilities of audiophiles, it can’t be any worse then their reading comprehension, if this is any example.

“As if the Placebo Effect explains away all positive results.”
It is in fact the standard against which positive results are measured.
“The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox”
Were that the case, It would be called the placebo fallacy. Trouble is, it really happens. It is not unusual to see 60 percent of patients improve when treated with a drug (”positive effect”), 5 percent to improve when not treated and 20 to improve when treated with a placebo.
“So, if Skeptics are not going to investigate the claims of controversial tweaks like good little scientists and just going to sit around in their soft loungers, then guess what? - talk really is cheap. Blah, blah and more blah.”
Yes, because skeptics aren’t the ones doing the talking. The null hypothesis requires no Devil’s Advocate. I one presents no evidence of guilt against a criminal defendant like a good little prosecutor, then guess what? He sits the innocent. When tests, such as audio gear, are not controlled, then the cause of a reviewer’s pleasure or displeasure in the experience cannot be confidently assigned to the hardware. When tests are done “in context,” they include factors that do not effect me, such as the reviewer’s room and chair and wallpaper and family and pets, etc.

“There is nothing more humorous than a Skeptic that can't shoot straight.”
Except perhaps someone who doesn’t appreciate skepticism in the proper sense and who shoots straight at targets that do not exist. A later comment about “!12 Angry Men” is telling. It is saying that jurors could not prove the defendant’s INNOCENCE, which they needn’t. It’s there already, just like the null hypothesis.

geoffkait's picture
Quote:

“As if the Placebo Effect explains away all positive results.”

It is in fact the standard against which positive results are measured.

“The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox”

Were that the case, It would be called the placebo fallacy. Trouble is, it really happens. It is not unusual to see 60 percent of patients improve when treated with a drug (”positive effect”), 5 percent to improve when not treated and 20 to improve when treated with a placebo.

The problem with your argument is that noone is saying the placebo effect never happens. Simply because the placebo effect might occur in some cases, for some listeners, logically it cannot explain away all positive results. Which is why Skeptics bring up placebo effect in the first place - to dismiss all positive results. Some positive results can be dismissed for other reasons, such as expectation bias, plain bad hearing, etc., but in the end there will be a significant number of positive results that cannot be ignored or explained away. That is precisely why the tweaks we're talking about are, in fact, controversial!

One should also consider that some audiophiles - let's call them advanced audiophiles - are more capable, through experience, of eliminating placebo type effects from their tests. The placebo effect should be lumped together with the false logic of, "believers in controversial tweaks must also believe in homeopathy and flying saucers."

Geoff Kait
Machina Dynamica

brindoporeso's picture

“The problem with your argument is that noone is saying the placebo effect never happens.”

Except perhaps you when you wrote, “The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox, along the lines of controlled blind testing.”

As you now admit, it IS more. It is a fact.

“Simply because the placebo effect might occur in some cases, for some listeners, logically it cannot explain away all positive results.”

Obviously. It defines their magnitude.

“Which is why Skeptics bring up placebo effect in the first place - to dismiss all positive results.”

Not the ones who do it right. In the drug scenario, it establishes the of the effect for which the drug is not responsible. If 20 percent of patients improve with a placebo and 60 with a drug, the drug deserves credit only for the 40 percent differential. If no skeptic ever brought up the placebo effect, no drug tests would ever be accepted for publication in scholarly journals.

“Some positive results can be dismissed for other reasons, such as expectation bias, plain bad hearing, etc., but in the end there will be a significant number of positive results that cannot be ignored or explained away. That is precisely why the tweaks we're talking about are, in fact, controversial!”

That should be precisely they they are not. Given proper experimental design, there’s the effect. Deal with it.

“One should also consider that some audiophiles - let's call them advanced audiophiles - are more capable, through experience, of eliminating placebo type effects from their tests.”

I’d just call them sane.

“The placebo effect should be lumped together with the false logic of, "believers in controversial tweaks must also believe in homeopathy and flying saucers."

Except that you just got through acknowledging that it happens, which it does.

geoffkait's picture

Fewer Errors
Submitted by brindoporeso on March 12, 2011 - 1:35am.

Quote:

“The problem with your argument is that noone is saying the placebo effect never happens.”

Except perhaps you when you wrote, “The fact is, the Placebo Effect is nothing more than some contrived tool in the Skeptics toolbox, along the lines of controlled blind testing.”

As you now admit, it IS more. It is a fact.

Duh! I think you must not have your thinking cap on as my point is the placebo effect, while it might affect some listeners, can be minimized or eliminated by careful testing. You know, eliminate - as in it doesn't exist! Placebo effect, like expectation bias, is simply another variable to be controlled. Like background noise. Thus, placebo effect is, in fact, nothing more than a distraction, a contrivance dreamed up by anti-tweak Skeptics. Furthermore, if we couldn't control such things, we would never be able to progress in this hobby since all audio components, speakers, cables and isolation devices, room treatments, etc. would suddenly fall prey to the placebo effect.

Quote:

“Simply because the placebo effect might occur in some cases, for some listeners, logically it cannot explain away all positive results.”

Obviously. It defines their magnitude.

It doesn't define anything. Well, maybe in the mind of the die hard Skeptic who won't take no for an answer. Once you eliminate all the listeners who are easily fooled, and the ones who don't know what they're listening to, the ones who get hoodwinked by the placebo effect, and those who have uncontrolled expectation bias, those who are hallucinating, then you have a set of valid positive tests. End of story.
In other words, it's OK to have a hundred negative results. As long as you have at least ONE positive result by someone who can actually hear under proper test conditions, you have a problem. A big problem.

“Which is why Skeptics bring up placebo effect in the first place - to dismiss all positive results.”

Not the ones who do it right. In the drug scenario, it establishes the of the effect for which the drug is not responsible. If 20 percent of patients improve with a placebo and 60 with a drug, the drug deserves credit only for the 40 percent differential. If no skeptic ever brought up the placebo effect, no drug tests would ever be accepted for publication in scholarly journals.[quote]

As I've already stated, noone does proper testing on these high dubious tweaks we're talking about. In fact, noone does much proper testing of anything as far as I can tell. I don't care about your drug scenario, it's irrelevant to audio, as I already stated. I realize die hard Skeptics think it's a valid argument. I think it's a phoney argument.

[quote

wrote:

“Some positive results can be dismissed for other reasons, such as expectation bias, plain bad hearing, etc., but in the end there will be a significant number of positive results that cannot be ignored or explained away. That is precisely why the tweaks we're talking about are, in fact, controversial!”

That should be precisely they they are not. Given proper experimental design, there’s the effect. Deal with it.

When a Skeptic gets around to devising a "proper test" for the Intelligent Chip, the Clever Little Clock, the Schumann Frequency Generator, the tiny singing bowls, or the Teleportation Tweak, then we can talk turkey. Meanwhile, talk is cheap. With respect to these highly dubious tweaks, a died in the wool Skeptic wouldn't know "proper experimental design" if one came up and pissed on his leg.

Quote:

“One should also consider that some audiophiles - let's call them advanced audiophiles - are more capable, through experience, of eliminating placebo type effects from their tests.”

I’d just call them sane.

I'd call them sane, too. I knew we'd finally agree on something. So, what's with all the melodrama and angst?

Quote:

“The placebo effect should be lumped together with the false logic of, "believers in controversial tweaks must also believe in homeopathy and flying saucers."

Except that you just got through acknowledging that it happens, which it does.

Of course I acknowledged it does. Why wouldn't I? It's not as if it's some secret knowledge only known by self-proclaimed Skeptics. Everyone is free to read "Zen and the Art of Debunkery." We agree that placebo effect is a real effect and that it affects some listeners. What does that have to do with the price of tomatoes? Now, if you are trying to say only novice audiophiles are affected by the placebo effect then, Bingo, we agree again! Can I assume you are a novice, or just a standard issue tweakophobe?

Geoff Kait
Machina Dynamica
Advanced Audio Concepts

brindoporeso's picture

“the placebo effect, while it might affect some listeners, can be minimized or eliminated by careful testing. You know, eliminate - as in it doesn't exist!”

Even when it does occur, it is removed by subtraction, as explained above.

“It doesn't define anything. Well, maybe in the mind of the die hard Skeptic who won't take no for an answer.”

And in the mind of every student of the philosophy of science who does not deserve to flunk out and of scientists who deserve publication. It defines exactly what I said it did. Again, I didn’t write the rules. I just report them. If you are not playing by the rules of chess, then you playing a game other than chess. So it is with science.

“When a Skeptic gets around to devising a "proper test" for the Intelligent Chip, the Clever Little Clock, the Schumann Frequency Generator, the tiny singing bowls, or the Teleportation Tweak, then we can talk turkey. . . . a died in the wool Skeptic wouldn't know "proper experimental design" if one came up and pissed on his leg.”

Skeptics, in the philosophical and not the psychopathological sense, are the ONLY people who know. For simplicity, I’ll refer to these as good and bad skeptics, respectively, the latter being the ones to whom you refer. Good skeptics defer to arithmetic, while bad ones are too stupid or to cowardly to do so. Suppose I play a round of golf with Tiger Woods and he shoot 70 and I shoot 90. Imagine someone not recognizing who won and wanting to call in a referee to render an opinion. Victory is settled by arithmetic, which is not subject to opinion. Suppose I said that I really should be the winner because I am a better singer or dancer or plumber. Sorry, not relevant. So it is with scientific theories. Whatever explains the data via the fewest ad hoc hypotheses wins. This does not make it THE TRUTH, just the winner, until something better comes along. This is one of the ways wrong is avoided in science.
If it quacks like a duck, then . . . it quacks like a duck. That does not mean it is a duck, but the theory that it is a duck is a better theory that it is an oak tree. I never says it’s a duck nor that it isn’t an oak tree. I let you listen and agree that it quacks like a duck, but then roots and bark and leaves are discovered and it turns out it really is an oak tree that makes a quacking sound when the wind blows through it. Someone might say, “See? You were wrong.” I wasn’t, because I never said it was a duck and not a tree. I accurately observed that the duck theory was the best explanation of the original data. “Proving the skeptics wrong” simply cannot be done to good skeptics because they are smart to arrange thing so that being wrong is simply not an option. Where uncertainty exists, the subjunctive mood is used rather than the declarative.
The main thing is to follow Ockham” Razor, which was originally given as an imperative that essentially says, “Don’t misbehave. Don’t explain wastefully. If it ain’t broke, don’t fix it. Once you get the golf ball into the cup, stop swinging.” Alas, even some scientists have a hard time properly understanding this, as when they mistakenly think it involves probability. Fortunately, they always seem to understand it within the context of golf. I’ve never heard anyone say that the golfer with the lowest score is USUALLY the winner or PROBABLY the winner or has the best CHANCE of being the winner. So it is in science. Theories are not true or false, they are better or worse. The ideal is to evaluate them exactly as in golf. In practice, it isn’t always quite so clean cut. But more often than one might expect, it’s nothing but arithmetic, which, again, is not subject to opinion.
“Good” skepticism ultimately underlies the presumption of criminal innocence, which does not preclude conviction, which happens all the time. When conviction occurs, it in no way “proves the skeptics wrong.” “Good” skeptics are the ones who know where to begin the process. It about showing one’s work. If you are brought to trial for murder and the prosecutor says, “I got nothin’,” you walk. Even without proving your innocence? Yes, not because you didn’t do it (maybe you did), but because those bearing the burden of proof didn’t show their work. Again, this would merely be an acquittal, not a declaration that you didn’t do it, so that if it was later found that you DID do it, it would not be true that the court was “wrong.”
Bad skepticism is also related to the Duhem-Quine thesis, which holds that you weasle your way out of any result by blaming the experiment. Falsificationism is the popular, post-war philosophical stance. It is popularly thought of as saying that theories cannot be proved, only disproved. Duhem-Quine holds that you can’t confidently disprove them either.

“I'd call them sane, too. I knew we'd finally agree on something. So, what's with all the melodrama and angst?”

We probably agree on much more, but I’ve been misperceived so much it’s been a struggle. It’s as if I say, “Paris is the capitol of France,” and the response comes back: “Oh. You’re one of those France-loving socialists!” Or I get, “The trouble with your argument is . . .” when I made no argument, but stated a fact. It’s like poised response triggered any stimulus, relevant or not, as when Lucy asks Ricky for shopping money, he reluctantly agrees and she says, “Why not?” Or when Sean Hannity presented a video in which Bill Mahr said, “Not all Republicans are bigots,” and then said, “So, he thinks all Republicans are bigots.” I refer to subtle tweaks and am accused of assume that they are all subtle, an assumption that would have prevented me from using the word “subtle” because it would then be as redundant as saying “unmarried bachelors.” All I ask is that the actual text be read. I am only responsible for what I wrote, not for what someone else imagined. Where there could be “trouble” with my arguments, I simply don’t make any. I am registered as a non-partisan voter because I never studied poli sci, so I don’t feel equipped to form valid political opinions. I wait for others to visit my areas of expertise and they often obligingly allow me to enjoy the thrill of victory.

“We agree that placebo effect is a real effect and that it affects some listeners. What does that have to do with the price of tomatoes?”

You had trouble expressing that consistently, as noted above. If this settles it, cool. As to the earlier business about controls, I’m glad you realize that it is sometimes hard to get people to appreciate them. I love when people observe that 70 percent of convicts came from broken homes or smoked marijuana. I ssay, yeah, and probably 99 percent of them grew up drinking milk and wearing shoes, which is an even better correlation. So where’s your outrage against milk and shoes? That’s what controls are for: to sort these things out.

“Can I assume you are a novice, or just a standard issue tweakophobe?”

I refer you to your own comments on “assuming” posted earlier. My specific relation to audio is trivial. I was speaking abstractly about general concepts. I’m sorry for any confusion with their application to specific audio matters. I was just hoping to provide some insight from the perspective of science, which was supposed to be the subtopic of this discussion. As for logic, if one consults the following resources, even if the terminology is new to one, the principles should be intuitively obvious:

www.nizkor.org/features/fallacies
leo.stcloudstate.edu/acadwrite/logic.html
usabig.com/autonomist/fallacies.html
www.csun.edu/~dgw61315/fallacies.html
www.fallacyfiles.org

geoffkait's picture
Quote:

And in the mind of every student of the philosophy of science who does not deserve to flunk out and of scientists who deserve publication. It defines exactly what I said it did. Again, I didn’t write the rules. I just report them. If you are not playing by the rules of chess, then you playing a game other than chess. So it is with science.

But isn't that exactly what the Bad Skeptics say, that science is on their side, and the laws of science cannot be disobeyed? "The candle of science in a world of demons," to paraphrase JA's Footnote to his article. The problem is noone can tell the Good Skeptics from the Bad Skeptics sicne they use the same arguments. Maybe we should require nametags. :-) Anything Skeptics, the Men of Science, take exception to are automatically placed in the box called Bad Science. As if all scientists think alike and agree about everything, even outrageous tweaks. That is precisely what this thread is all about. By claiming one is a Man of Science it automatically means that anything HE deems UNscientific be labeled False or a Fraud. Isn't that simply demonizing it? You might as well say it's paranormal, or pychological or delusional. You're the expert on logic, but isn't that argument an Appeal to Authority? In the specific case of JA, a PhD Theoretical Physics, anything that he's not OK with science-wise, physics-wise, must fall under the category, Hogwash. E.g., Intelligent Chip, Brilliant Pebbles, etc. Is that not the point of JA's article - that people in the industry should be more skeptical of "outrageous tweaks" - the ones that give so-called scientific types heartburn, the ones JA has been wringing his hands about for the last six years. So does that make JA a Good Skeptic or a Bad Skeptic? - you tell me, you're the expert.

Raising any scientific truism (such as the placebo effect) in the defense of science, when attacking preposterous sounding tweaks, such as the Intelligent Chip or Brilliant Pebbles is always a good strategy in the war on hard to swallow tweaks since the Laws of Physics cannot be disobeyed; if you make it appear you're solidly on the side of physics, in fact ALL science, how can you lose? LOL

And just exactly how do you propose one go about testing an audio device that purportedly has no effect on the audio signal or the acoustic wave reaching the listener's ears?

I reiterate, whether the person is a Good Skeptic or Bad Skeptic, I have yet to see ANY Skeptic actually do ANYthing except construct clever arguments. One never sees ANY of these Skeptics actually roll up his sleeves and investigate these outrageous Science-disobeying devices, the objects of so much writing. Isn't investigation really the most important part of true skepticism? I suspect most Skeptics just want to appear to be on the side that's winning, sitting comfortably in the peanut gallery.

Quote:

“When a Skeptic gets around to devising a "proper test" for the Intelligent Chip, the Clever Little Clock, the Schumann Frequency Generator, the tiny singing bowls, or the Teleportation Tweak, then we can talk turkey. . . . a died in the wool Skeptic wouldn't know "proper experimental design" if one came up and pissed on his leg.”

Skeptics, in the philosophical and not the psychopathological sense, are the ONLY people who know.

I gather you are unfamiliar with the devices I mentioned, otherwise you probably would have taken a more precautionary stance. But I can certainly understand if you are arguing from the position of a Good Skeptic, not an audiophile. What is needed is a real, Good Skeptic who understands what he's up against scientifically and audiophile-wise, an experienced open-minded listener, when it comes to these outrageous tweaks we're talking about. So far, I don't see that happening. I do, however, see people posing as Good Skeptics, perhaps also posing as audiophiles, using illogical arguments to attack any errant sympathizers of high dubious tweaks. One wonders what can their motive be - to illuminate mankind? LOL

Geoff Kait
Machina Dynamica

brindoporeso's picture

I apologize for any confusion that may have arisen, which I will try to resolve. Commonly, the word “skepticism” refers to a form of misbehavior. The people you call skeptics are either idiots or cowards or both. Unfortunately, philosophers use similar terminology when discussing a form of behavior that is logically proper and necessary. Without it, scientists would not be doing it right and science would not be happening. So, I’ll refer to your skeptics as “idiots” and will contrast them with “scientists.” I could qualify these terms to make them more precise, but I’ll forego that in the absence of further confusion. After this, I may retreat to the philosophical realm where my terminology has been canonical since long before I showed up and causes now confusion. So. I’ll say good bye here and wish you well.

“The problem is noone can tell the Good Skeptics from the Bad Skeptics sicne they use the same arguments.”

Idiots must eventually resort to bogus ones. Editors don’t decide what gets published in the scholarly literature based on a coin toss. Standards exist, and upward-looking snobbery (the pot calling the sugar black) represents just such desperation.

“You're the expert on logic, but isn't that argument an Appeal to Authority?”

Only if I appeal to myself, which I don’t. If logic isn’t intuitively obvious to you, then you have no other option than appeal to authority and have no way to know if you’re right or wrong. It’s supposed to be about recognition, not trust. I even advised looking past Aristotle. The goal is to be smart enough to make such evaluations oneself. Of course, this can be taken to sociological extremes. When friends talk about people who make more money than I do (watch out for that argumentum ad crumenam) I say, “Yeah. Know who else made more? Lassie, but money did not earn Lassie the executive pole of the dog leash.” If I say “2+2=4” and you can’t tell if I’m right or wrong, what is there to say except, “Fetch!” You mention the need for open-minded listeners. The mind of the scientist is open, but not empty. It knows what to do with what goes in.

“And just exactly how do you propose one go about testing an audio device that purportedly has no effect on the audio signal or the acoustic wave reaching the listener's ears?”

Again, I’m talking abstractly and I have no dog in this fight. I merely quote rule books that I did not write (though I could have because I understand them). If I said “2+2=4” and you said “2+2=5,” you’d be disagreeing not with me but with arithmetic. I’m just the messenger. Not only is it futile to kill the messenger, I’ve been witnessing the killing of other messengers with other messages. No problem. Not even sticks and stones can break my bones when hurled not at me but at a straw man (or at least a different one). No harm, no foul. (The consolation was the delicious irony in your comment about MY bad aim.)

“One never sees ANY of these Skeptics actually roll up his sleeves and investigate these outrageous Science-disobeying devices, the objects of so much writing.”

See my original post. Whether talk is cheap or not, when there is none at all, some people misbehave. When selecting a jury for a criminal case, lawyers like to ask prospective jurors what verdict they would render if asked to do so before the trial was conducted. Any answer other than “not guilty” should get one disqualified because it demonstrates a lack of understanding of the system. Where there is no talk, there is acquittal. Where there is no talk, the null hypothesis persists. You expressed doubt that certain people would ever design a proper test for something. Whether talk is cheap or not, the null hypothesis is simply the observation that until SOMEBODY, ANYBODY designs and performs such a test, there is nothing about which to talk. Irresponsible impatience in such situations gets people denied publication in the scholarly literature, though this may not be such a significant issue on the recreational or commercial level.

“I suspect most Skeptics just want to appear to be on the side that's winning.”

Perhaps. The goal is to know what constitutes victory. In science, it’s often as easy as in golf, hence my earlier illustration.

billmilosz's picture

Naturally, Stereophile is pointing out that you can't PROVE that NO ONE can hear A/B differences between two TOSLINK interconnects (for example) - and in terms of pure logic this is true. You can't "prove a negative" in this way.

But we don't live in a world of Platonic ideals and pure logic. We live in a PRACTICAL subset of such purity. So I say, if it has not been DEMONSTRATED IN OBJECTIVE BLIND TESTS that audible differences exist between two TOSLINK cables (for example) then I am not willing to SPEND MY MONEY on such goods! It may be logically correct that you cannot prove that NO differences exist, but until your prove in a practical way that audible differences DO exist, I'll hang on to my money, thank you very much.

Stereophile has a vested interest in protecting their subjectivist position, and so, ipso facto, they are not to be trusted.

But, if you DO buy their line, please contact me- I have a bridge for sale in Brooklyn, and if you place your speaker wire on this bridge for at least 24 hours it will increase the resolution and soundstage of your system. (Go ahead, prove that it won't!)

John Atkinson's picture
billmilosz wrote "Naturally, Stereophile is pointing out that you can't PROVE that NO ONE can hear A/B differences between two TOSLINK interconnects (for example) - and in terms of pure logic this is true. You can't "prove a negative" in this way."

That is correct. And certainly it is possible to measure differences between 2 Toslink cables, in terms of bandwidth, which in turn lead to different amounts of jitter, depending on what D/A processor is in use. Audibility of the effects of that jitter will also depend on the D/A in use.

"Stereophile has a vested interest in protecting their subjectivist position, and so, ipso facto, they are not to be trusted."

With respect, this seems a logical non sequitur, given that the essay that triggered your comment and that questioned the effectiveness of unlikely-sounding tweaks was indeed published in Stereophile. So by your own argument, Jim Austin's essay is itself not to be trusted. :-)

John Atkinson
Editor, Stereophile

geoffkait's picture
Quote:

But we don't live in a world of Platonic ideals and pure logic. We live in a PRACTICAL subset of such purity. So I say, if it has not been DEMONSTRATED IN OBJECTIVE BLIND TESTS that audible differences exist between two TOSLINK cables (for example) then I am not willing to SPEND MY MONEY on such goods! It may be logically correct that you cannot prove that NO differences exist, but until your prove in a practical way that audible differences DO exist, I'll hang on to my money, thank you very much.

Looks, it's really very simple. The ONLY real reason to purchase a piece of audio gear or cable or accessory is if it's better than what you already have in terms of sound quality. Now comes the difficult part. You have to obtain the piece of gear or cable or accesssory and insert it into your system. Only then can you evaluate its effect on the sound. Who, especially someone who's afraid of making a big mistake, would take someone else's word for the sound of an amp, a speaker, a cable or a tweak? You have heard of 30 day trial periods, haven't you? These demands for proof sound eerily like the lone holdout in 12 Angry Men, "But you can't PROVE it!"

Look, when you go out to purchase a TV do you conduct controlled blind tests to figure out which brand to buy? Of course you don't. That would be ridiculous, right? You pick the TV that has the best picture at your price point. That's what we all do. No data, no explanation of operation, no controlled blind tests. That would be ridiculous, right? Now, how is picking audio gear or cables or tweaks any different? Isn't it much more logical and PRACTICAL to pick audio stuff according to how it sounds rather than rely on somebody out there conducting controlled blind tests (which - for your information - noone ever does, anyway). LOL

Cheerio

Geoff Kait
Machina Dynamica

LytleSound's picture

Acoustics is the oldest field within Physics, which, in turn, is the oldest field of the sciences. Within acoustics falls both electroacoustics and psychoacoustics. Statistics is a tool for both of these sciences; applied more for the latter than for the former. When it comes down to determining if system A sounds different from system B, the realm of psychoacoustics has been entered and an entire set of statistical issues arrive.

Issue 1: for whom must system A be different from B? If trained/experienced listeners are used, the perception of the outcome may be different. So-called golden ears are usually experienced listeners for the type of comparison being conducted. Inexperienced listeners, actually harder to define, are the "man off the street." Issue 2: to what level of statistical significance should this difference be required to meet? This is typically set at the 90% level for behavioral work, meaning that the outcome has a 90% likelihood of being repeated if the exact same comparison methods were to be repeated. Issue 3: what statistical power is required? This is an illusive concept to many, but can be distilled to the issue of the number of listeners to use in the study to determine the expected outcome at the level of significance selected and is usually based on the results of a smaller pilot study. Issue 4: will the test be blind or open? If blind, an ABX comparison may be performed, if open, only an AB comparison may be performed. Issue 5: will the test be sequential, relying upon auditory memory, or immediate, where both system A and B are present? Auditory memory is assumed to be poor for the untrained listener and reputed to be good for the golden ear.

Seldom have all of these issues been addressed for audiophile concerns. And, yet the outcomes of those comparisons that have been done "scientifically" (which really means that some level of statistical rigor was applied) are either held to by proponents or denied by opponents, almost independent of the comparison itself.

Notice that I haven't asked about the issue of what is better or worse, just if there is a difference. Also, going to Issue 1, I don't remember a comparative study of golden ears to inexperienced listeners. I have experience with golden ears in an industry where a golden ear was required to listen to every product before it was shipped. Regardless of what any other test showed, the golden ear's decision could not be overruled. The golden ear most often could point out the flaw so that the product could be fixed before being shipped - after once again getting by the same golden ear - or scrapped. None of these golden ears purported to be audiophiles; they weren't culled from a group of audiophiles nor did most even listen to "music on the hifi." And, again, their criteria weren't better versus worse or same versus different, but pass versus fail.

By the way, Acoustics is also among the least understood fields in science, particularly as compared to another branch of physics such as Optics.

geoffkait's picture
Quote:

Acoustics is the oldest field within Physics, which, in turn, is the oldest field of the sciences. Within acoustics falls both electroacoustics and psychoacoustics. Statistics is a tool for both of these sciences; applied more for the latter than for the former. When it comes down to determining if system A sounds different from system B, the realm of psychoacoustics has been entered and an entire set of statistical issues arrive.

You said, "Acoustics is the oldest filed within physics." It's interesting to compare/contrast that statement, which I'm not doubting, to your last statement, "Acoustics is also among the least understood fields in science," which I also don't doubt, but find ironic and analogous to audiophiles who begin their sentences with, "I've been in this hobby for 40 years so you can believe me when I say...."

Quote:

Issue 1: for whom must system A be different from B? If trained/experienced listeners are used, the perception of the outcome may be different. So-called golden ears are usually experienced listeners for the type of comparison being conducted. Inexperienced listeners, actually harder to define, are the "man off the street." Issue 2: to what level of statistical significance should this difference be required to meet? This is typically set at the 90% level for behavioral work, meaning that the outcome has a 90% likelihood of being repeated if the exact same comparison methods were to be repeated. Issue 3: what statistical power is required? This is an illusive concept to many, but can be distilled to the issue of the number of listeners to use in the study to determine the expected outcome at the level of significance selected and is usually based on the results of a smaller pilot study. Issue 4: will the test be blind or open? If blind, an ABX comparison may be performed, if open, only an AB comparison may be performed. Issue 5: will the test be sequential, relying upon auditory memory, or immediate, where both system A and B are present? Auditory memory is assumed to be poor for the untrained listener and reputed to be good for the golden ear.

I'm actually a big fan of auditory memory and find it rather surprising that others have trouble in this area. Oh, well. I do find that repeated testing or long listening sessions can get tiresome, however. One thing that probably should be mentioned is the Audio System used for testing. Do you have an idea how far one must go to find a system one can tolerate listening to for more than 5 minutes? - a system that is not overwrought or distorted, especially at higher volumes, or hideously thin and threadbare, with no resemblance to a high end system? Who is to be The Arbiter of Sound for the Test System - a cynical, all-thumbs tweakophobe with an ax to grind? LOL

Quote:

Seldom have all of these issues been addressed for audiophile concerns. And, yet the outcomes of those comparisons that have been done "scientifically" (which really means that some level of statistical rigor was applied) are either held to by proponents or denied by opponents, almost independent of the comparison itself.

I dunno, seems to me that statistics and data have been overdone. An obvious example is the old Japanese amplifiers of yore with great THD specs that didn't fair too well at all when it came to actually listening to them, whereas tube amps with Order of Magnitude higher THD are much more "musical" and actually sound less distorted. And how do you propose testing such things as Intelligent Chips, Cleevr Little Clocks, Schumann Frequency Generators, Mpingo Discs, crystals, hairdryers with tourmaline?

Quote:

Notice that I haven't asked about the issue of what is better or worse, just if there is a difference. Also, going to Issue 1, I don't remember a comparative study of golden ears to inexperienced listeners. I have experience with golden ears in an industry where a golden ear was required to listen to every product before it was shipped. Regardless of what any other test showed, the golden ear's decision could not be overruled. The golden ear most often could point out the flaw so that the product could be fixed before being shipped - after once again getting by the same golden ear - or scrapped. None of these golden ears purported to be audiophiles; they weren't culled from a group of audiophiles nor did most even listen to "music on the hifi." And, again, their criteria weren't better versus worse or same versus different, but pass versus fail.

By the way, Acoustics is also among the least understood fields in science, particularly as compared to another branch of physics such as Optics.

As I stated above, what's missing is the audio system that will be used for the test - who's assembling it, who decides what it contains, what room treatment, if any, is used, what isolation devices, if any, are used and who decides whether the sound achieved for the Test System is worth anything for purposes of testing?

Sincerely,

Geoff kait
Machina Dynamica
We Do Artifical Atoms Right

Jorgensen's picture

Geoff - homeophathy is not most commonly a term employed by self termed sceptics, as you write. Unfortunately, it is much more commonly used by homeopathists trying to pass off their bogus philosophy to those who hope for a cure for their ailments. This should be prohibited by law, just like quackery is today.

I haven't done any research in homepathy, as you suggest I might should have done before complaining about it. However, this wouldn't be necessary, as the uselessness of the intervention has already been extensively documented, as summed up in a meta-analysis in Lancet in 2005 (Shang et al. Lancet 2005; 366(9487):726-32). I have, however, done lots of other research. I am 35 years old, but you will find 38 'hits' to my name on PubMed, including in such journals as BMJ, Lancet Oncology and the New England Journal of Medicine. I teach scientific theory at both university and post graduate level. You do not have to do research in a certain topic to form an opinion about it. You just have to read the literature, if one of reasonable quality exists.

What I aimed at in my text when I referred to surprinsing results was not whether exotic tweaks would fail in a double blinded test. I was thinking about how some people might become disappointed in their ability to hear a difference between two audio components, and therefore also regret large investiments. I agree that this was not stated clearly enough in my text, but I can also see how you would be likely to respond in the way you did, having seen your web page about the 'Brilliant Pebbles'. I encourage all sterophile readers to do the same - it is deeply fascinating: http://www.machinadynamica.com/machina31.htm.

Let me remind you that science tends to select the most plausible hypotheses to test first.

geoffkait's picture
Quote:

I haven't done any research in homepathy, as you suggest I might should have done before complaining about it. However, this wouldn't be necessary, as the uselessness of the intervention has already been extensively documented, as summed up in a meta-analysis in Lancet in 2005 (Shang et al. Lancet 2005; 366(9487):726-32). I have, however, done lots of other research. I am 35 years old, but you will find 38 'hits' to my name on PubMed, including in such journals as BMJ, Lancet Oncology and the New England Journal of Medicine. I teach scientific theory at both university and post graduate level. You do not have to do research in a certain topic to form an opinion about it. You just have to read the literature, if one of reasonable quality exists.

I agree that this was not stated clearly enough in my text, but I can also see how you would be likely to respond in the way you did, having seen your web page about the 'Brilliant Pebbles'. I encourage all sterophile readers to do the same - it is deeply fascinating:

http://www.machinadynamica.com/machina31.htm.

Let me remind you that science tends to select the most plausible hypotheses to test first.

....................................................

I also encourage more Stereophile readers to see my page about Brilliant Pebbles. I find the subject fascinating, too. And I often encounter folks with expertise in some area or another who express strong reactions to the idea of using crystal structures for audio applications. LOL The theoretical page for Brilliant Pebbles is at:

http://www.machinadynamica.com/machina17.htm

You are most likely correct - you do not have to do research on a certain topic to form an opinion about it. In fact, you apparently don't even have to review the available literature. That has been made abundantly clear by JA in his article (the subject of the OP) as well as innumberable times on this and many other audio forums.

Cheers,

Geoff Kait
Machina Dynamica
Advanced Audio Concepts

0pteron's picture

The topic and the discussion are both quite stimulating. I would like to offer a possible explanation for the reluctance of many folks to view science with the respect it might otherwise deserve:

So far as I know, there is no test that measures the various apparent dimensions of a soundstage. So far as I know, there is no test that tells me whether a given sound is that of a trumpet, oboe, cornet, violin, viola or flute playing a given note.

Until such time as these tests are available, there is a gap between what is being measured and what I, and I believe a lot of other people, are interested in.

It is this kind of test result that will tell me whether or not a cable or other component makes a difference.

When these tests are used, I will be delighted to pay more attention to science. Until then, the contribution of science is intersting, but not nearly as important as my ears in evaluating equipment and recordings.

Cheers.

Jon

John Atkinson's picture
0pteron wrote: "So far as I know, there is no test that measures the various apparent dimensions of a soundstage. So far as I know, there is no test that tells me whether a given sound is that of a trumpet, oboe, cornet, violin, viola or flute playing a given note."

Exactly so. That is because all these are constructed by the brain based on the information that reaches the two ears. In that sense they are not real. The reality is two audio-bandwidth pressure waves that vary with time emitted by the two loudspeakers. The "oboeness" or "trumpetness" etc, and where the acoustic objects that represent those instruments are located within the soundstage, are constructs within the listener's brain and thus are not available for measurement.

I examined this subject in the introduction to my January 1992 review of the Westlake BBSM-6F loudspeakers - see www.stereophile.com/standloudspeakers/192west .

John Atkinson
Editor, Stereophile

Jorgensen's picture

John - I agree that "The "oboeness" or "trumpetness" etc, and where the acoustic objects that represent those instruments are located within the soundstage, are constructs within the listener's brain and thus are not available for measurement", in the sense that "meassurement" here means measurement using a microphone and a computer or similar. But this does not affect whether we can or should meassure differences by comparing audio components in a blinded fashion, or if this type of testing is scientific and in that sense more reliable than unblinded testing.

Let me start by stating what I am not saying. I am not saying unblinded tests are useless. I read Sterephile with great interest and feel confident that you have very capable reviewers that have the great advantage of having heard lots of equipment that I have never heard. Some of them may even be equipped with the famous 'golden ears'.

There is no doubt that someone looking to buy an audio component would be interested in tests that were as reliable as possible. I think blinded testing is generally more reliable than unblinded testing, particularly when it comes to 'soft' outcomes that can be influenced by psychology and preconceived notions, such as is the case in hi fi, whether we like it or not.

This is supported by good research and is one of the reasons that blinded wine tastings are so much fun (another being the alcohol, of course). Being told that a wine is very expensive will, on everage, make people evaluate it as better tasting than if they are told it is cheap. You could call this a placebo effect or say that the reason the wine tastes better doesn't really matter, as long as you get the good experience. But if I found out that I had really just bought a a case of cheap wine at a high price, I would still feel cheated.

I therefore see blinded tests as a potentially valuable supplement for Stereophile that could put it in a league of its own among the plethora of hi fi mags in this world.

geoffkait's picture

The fact that tests are Blind has precious little to do with how reliable they are. Much of the reliability and success of blind tests, like any type of test, have to do with experience/competance of the tester, resolution/competance of the audio system used for testing as well as less definable or even unknown factors. Besides, there are no Standards for blind tests, and even proponents of blind tests disagree on the protocols that should be employed. So it goes....

Geoff Kait
Machina Dynamica

Jorgensen's picture

I'm sorry Geoff, but this is where you are mistaken. Blinding is pivotal for reliable evaluations of subjective outcomes, such as when testing a new versus an old painkiller. It really isn't just my opinion. It is a mainstay of scientific testing and the effect of blinding in this context is extensively documented in the scientific litterature. Those who are not blinded will substantially over-estimate the effect of interventions aimed to change subjective outcomes. This is why blinding is required in guidelines describing how to test subjective outcomes and why no respectable medical journal will accept a study that did not use blinding, if this was reasonably possible (blinding isn't always possible, e.g. when comparing surgery to medication). Even the best scientist (or most experienced stereophile reviewer)can be influenced in their judgement of effect by looks, price, or expectations. You can add any number of factors to this list. This is not a flaw on their behalf. It is basic human nature. But we can do something to minimise this effect: blind those who assess.

To improve reliablility of testing hi fi equipment, it would be a valuable addition to a regular review if the components were sent before a blinded panel, to see if the finding of the the experienced reviewer with all the background knowledge could be confirmed. Much like when you send something for a lab test.

geoffkait's picture

There you go again with the medical analogy silliness. That analogy has been used by proponents of blind testing since Christ was in daipers and it's still not a convincing argument. As if we're supposed to respect the medical profession, that's a laugh. The only reliable way to find out how a component, speaker or any audio device is going to sound in your system is to get a hold of one and insert it into your system. Then you can blind test until the cows come home. Everything else is a pig in a poke. Why would you take someone else's word for anything, anyway? That's what I'd like to know.

If blind testing is as important as you claim how come no one actually uses it, I mean except for a few eccentrics? Good luck with all those medical journals. :-)

"There are none so blind as those who will not see." - old audiophile axiom

Geoff Kait
Machina Dynamica

Jorgensen's picture

Goeff - I wonder why you you read Stereophile at all? You write that:

"The only reliable way to find out how a component, speaker or any audio device is going to sound in your system is to get a hold of one and insert it into your system. (...) Everything else is a pig in a poke. Why would you take someone else's word for anything, anyway? That's what I'd like to know".

If this was the case, there would be no justification for hi fi journals, blinded testing or not.

Audio components do have specific characteristics and qualities that can be described in a meaningful way to help those in the market for such things, keeping in mind that their importance is partly reliant on the context in which they are used (I think anyone would agree with you on that). If you do not accept this, everything stops.

I can easily understand why you wouldn't trust the medical profession. Too many scandals have come from thoughtless collaboration with big pharma. But what I am asking you to trust is science, not the medical profession. I do not simply claim that blinding is important. I point out that there is solid documentation for this. It is a fact in the same way that evolution is i a fact. I hope you do accept evolution, despite examples of constructed 'missing links' by misguided scientists.

I can think of several reasons that blinded testing in hi fi has not gained acceptance.

1: It is uncomfortable to be proven wrong. It is much nicer to remain within your frame of understanding of this world, especially if you have made a career of selling your subjective judgements (no offense meant, I appreciate that skilled people have done this).

2: It could threaten the existence of entire audio businesses (none mentioned, none forgotten). If trained listeners cannot identify a difference in a blinded test, is it an important difference at all, if one exists?

3: It could threaten income from advertising for hi fi journals.

4: Most audio buffs are not scientifically trained and do not know how easy it is to be influenced by context when making subjective judgements.

5: Tradition in the field.

This list can be expanded ad nauseam.

geoffkait's picture
Quote:

Goeff - I wonder why you you read Stereophile at all? You write that:

"The only reliable way to find out how a component, speaker or any audio device is going to sound in your system is to get a hold of one and insert it into your system. (...) Everything else is a pig in a poke. Why would you take someone else's word for anything, anyway? That's what I'd like to know".

If this was the case, there would be no justification for hi fi journals, blinded testing or not.

Good point. I didn't think of that. Heh heh

Quote:

Audio components do have specific characteristics and qualities that can be described in a meaningful way to help those in the market for such things, keeping in mind that their importance is partly reliant on the context in which they are used (I think anyone would agree with you on that). If you do not accept this, everything stops.

As I stated previously, the only meaningful way to determine how a particular audio product will perform, and if it will perform to your satisfaction is to obtain it and see how it works out in your system - with all the myriad variables of your room and your system and your ears. Isn't that really the only context that matters?

Quote:

I can easily understand why you wouldn't trust the medical profession. Too many scandals have come from thoughtless collaboration with big pharma. But what I am asking you to trust is science, not the medical profession.

Geez, where have I heard that before? Oh, yeah, in JA's article, As We See It, the subject of this thread -- "Trust Science." Ha Ha

Quote:

I do not simply claim that blinding is important. I point out that there is solid documentation for this. It is a fact in the same way that evolution is i a fact. I hope you do accept evolution, despite examples of constructed 'missing links' by misguided scientists.

I hope you're not trying to say that one must accept evolution to be a good, scientific audiophile. Next you'll be telling us audiophiles have to accept Catholicism. There's a lot of documentation for that, too. I suppose those who don't accept blind testing are condemned to eternal darkness. Ha Ha

Quote:

I can think of several reasons that blinded testing in hi fi has not gained acceptance.

1: It is uncomfortable to be proven wrong. It is much nicer to remain within your frame of understanding of this world, especially if you have made a career of selling your subjective judgements (no offense meant, I appreciate that skilled people have done this).

Unfortunately, when results of blind tests are negative and it appears Science has saved the day, guess what? - the test was not performed properly by the proper person under the proper conditions. Thus, just like any test, Blind Tests don't necesarily prove anything. You can't hang your hat on them. There are no standards for blind tests. Everyone is just stumbling around in the dark. Is it possible that promoters of blind tests have something to sell, too?

Quote:

2: It could threaten the existence of entire audio businesses (none mentioned, none forgotten). If trained listeners cannot identify a difference in a blinded test, is it an important difference at all, if one exists?

Sorry, it's already been demonstrated many times that they can't identify a difference in a blind test, at least reliably. If your statement were true The Amazing Randi would be in the poorhouse now. Someone should come up with the definitions of "a trained listener" and "an important difference in the sound."

Quote:

3: It could threaten income from advertising for hi fi journals.

It's more likely promoting something so frought with contradictions and controversy as blind testing would threaten income from advertising for hi fi journals.

Quote:

4: Most audio buffs are not scientifically trained and do not know how easy it is to be influenced by context when making subjective judgements.

I suspect you have it backwards. The scientifically trained audiophiles realize how suspect blind tests are for the reasons I've stated. And the trained listeners among us sympathize with those who are unsure of their listening ability, the ones who need someone else to lead them by the ear, so to speak.

Quote:

5: Tradition in the field.

Hmmmm.... is that anything like the tradition of dunking witches in water until they drown? Ha Ha

{quote]This list can be expanded ad nauseam.[/quote]

I'm sure you're correct. I've read Zen and the Art of Debunkery, too.

Geoff Kait
Machina Dynamica
Advanced Audio Conceits

c1ferrari's picture

Good read, Jim. Thank you.

brindoporeso's picture

Jim Austin brings up the issue of "‘trusting your ears,’ despite abundant evidence from neuroscience and cognitive psychology that our ears and other senses, though supremely acute, are supremely unreliable.” On the subjective level, this unreliability is not necessarily a problem. We have all experienced optical illusions in which we perceive patterns and objects that demonstrably have no existence outside of ourselves, yet the perception really happens. Doubt arises only concerning external reality, not our perceptions. The philosopher Daniel Dennett points out the absurdity of saying, “I think I may be in pain, bit I’m not sure.” What’s to doubt?

Also, a word of caution: Although one is perfectly correct not to expect anything from “a jar of pretty rocks,” that is not the real issue. Penicillin was initially derived from a glop a ugly mold. Politicians often ridicule spending money on research involving what they consider silly organisms, even though modern genetics derives from the study of fruit flies. The point is that both pretty rocks and ugly mold are subject to the same logical constraints. Popperians utilize “Conjecture and Refutation,” such that anything can be the source of theories: (dream, drug-induced hallucination, etc.). But theories must then stand up to the necessary logical tests.

Having finally visited the website devoted to “Brilliant Pebbles” (BPs), I can now apply the general principles discussed earlier and make some concrete comments. Of the issues arising in the paper, I will focus on one. I am not a merchant and have nothing to sell. I merely wish to offer an example of the sort of scrutiny and criticism to which I have been subject in my world.

Suppose a sugar pill with blue dye costing $100 per pill benefitted 20 percent of patients, and that a sugar pill with red dye costing 2 cents per pill similarly benefitted 20 percent of patients. The claim that the blue dye was the cause of the effect would not be supported. The effect of the blue pill is fully explained by the simple fact that it is a pill. That is, it is fully accounted for by its “pillness.” It is only by exceeding this threshold that something would deserve to be called a drug. Also, the difference in price is not justified by a difference if performance. Failure to outperform a placebo earns a candidate drug no medals. If an expensive treatment works, but an inexpensive placebo works just as well, it allows patients to enjoy the effect while simultaneously saving money.

The principle issue regarding BPs is that the discussion, as presented on the visited website, offers an explanation for why they would outperform generic, amorphous stones, but no report of the use of the control that would isolate this factor.

The difference in performance between BPs and generic stones is either statistically significant or it is not. If it is, then the null hypothesis is falsified and mere “stoneness” or ”petrosity” is shown to be insufficient to account for the observed effect. If the difference is not significant, then the null hypothesis stands unfalsified and the effect of BPs is explained by their “stoneness,” which is also a property of generic stones that are available in vacant lots at a substantial discount.

Which is the case? The answer may be published elsewhere, but there it is no report of it on this particular website. In the absence of this control, there exists only an explanation in search of an explanandum. Thus, this report taken in isolation offers little of interest to science. (Its value as poetry is left to the appraisal poets.) This is not to say that BPs do not work. But even if they do, this is really only worth knowing when it is also known that ordinary stones do not. A theory is offered from which is derived the expectation that ordinary rocks would not work, but this remains an untested hypothesis, not empirical data. Woulda-coulda-shoulda is not a valid category in logic any more than it is in Simon Says. This indeterminacy leaves the null hypothesis on the throne, yawning and trimming its fingernails. Ludwig Wittgenstein reformulated Ockham’s Razor to say that whatever is explanatorily unnecessary is meaningless. A defense attorney should have to fix what ain’t broke only when the jury is too stupid to recognize when it ain’t broke. Competent jurors and scientists need no such help. Exerting effort to get to where one already sits (fait accompli) is, as per Wittgenstein, unnecessary, wastefully and meaningless.

This should mitigate possible concerns about veracity. Even if this BP business is all a lie, it is an insufficient one, such that even if it is all true, it is still not enough. It takes the form of an accent fallacy, as if one said, “Jim Austin was sober today.” Perhaps he always is. “These stones work” may be a true statement, but it raises the question: “Which ones do not?” Crystaline structure per se can be found in table salt, a bag of which should be tried before resorting to anything more expensive in case its crystals turn out to be fortuitously tuned for this application.

Health insurance providers do not pay for experimental treatments because they are waiting for the other shoe to drop. In science, ignorance of the existence of that other shoe is intolerable. Simply being “pretty rocks” is not grounds for derision, but if confession were to reveal this to be a hoax, grounds for surprise would be lacking.

Little controversy arises about all this on the superficial level of commerce: a commodity is produced, people pay money for it, end of story. Merchants are not always required to show their work exhaustively and science is not always the issue. In a recent documentary, when astronaut Frank Borman was asked about the science objectives of the Apollo 8 mission, he replied, “Science?!” Additionally, the occupancy of hot kitchens (such as peer-reviewed scholarship) is voluntary. (“Doc, it hurts when I do this.”) However, when science is the issue, incomplete experimentation and testimonials submitted by persons of uncertain ontological status are not enough to command scholarly attention.

With regard to the Jorgensen exchange: The null hypothesis is not an opinion, and was in force prior to the cited Lancet paper. The report of no effect does not change the situation from ignorance to enlightenment. It reports the situation to be unchanged. Upon acquittal, the status of a criminal defendant does not change to innocence from something else. Jorgensen’s students need to understand these things if they hope to qualify, respectively, for science or jury duty. Whatever one should or should not do prior to forming an opinion, my students would not be taught to form opinions but to recognize facts.

geoffkait's picture
Quote:

Jim Austin brings up the issue of "‘trusting your ears,’ despite abundant evidence from neuroscience and cognitive psychology that our ears and other senses, though supremely acute, are supremely unreliable.”

Yes, and isn't it ironic that it's JA who brings up the issue since he's one of those who actually doesn't trust his ears, you know, what with all the demands for data and meaurements? Nor has he ever heard the BP demonstrated or even seen them, for that matter, maybe in a picture LOL. That's a favorite argument of skeptics, especially skeptics of outrageous and controversial tweaks, one right out of Zen and the Art of Debunkery.

Quote:

On the subjective level, this unreliability is not necessarily a problem. We have all experienced optical illusions in which we perceive patterns and objects that demonstrably have no existence outside of ourselves, yet the perception really happens. Doubt arises only concerning external reality, not our perceptions. The philosopher Daniel Dennett points out the absurdity of saying, “I think I may be in pain, bit I’m not sure.” What’s to doubt?

Fortunately for experienced audiophiles, optical illusions have precious little to do with home audio reproduction unless, of course, one's not really sure what he's listening to, you know, doesn't trust his ears! Noone is saying only one test is always sufficient; sometimes these things are more difficult to pin down so multiple tests might be necessary, perhaps using a different system for the test might be necessary. More negative or ambiguous results have been obtained on bog standard systems than this world dreams of. Ha Ha So, it's not really so much of a philosophical issue, as you seem to think, but more of the old goldfish in a fishbowl issue, if you see what I mean.

Quote:

Also, a word of caution: Although one is perfectly correct not to expect anything from “a jar of pretty rocks,” that is not the real issue. Penicillin was initially derived from a glop a ugly mold. Politicians often ridicule spending money on research involving what they consider silly organisms, even though modern genetics derives from the study of fruit flies. The point is that both pretty rocks and ugly mold are subject to the same logical constraints. Popperians utilize “Conjecture and Refutation,” such that anything can be the source of theories: (dream, drug-induced hallucination, etc.). But theories must then stand up to the necessary logical tests.

Thanks for the warning. LOL
I certainly agree that things should stand up to tests, who wouldn't? Seems like a reasonable statement. But it may be best if the tester is someone who hasn't made up his mind already, eh? and someone capable of assembling a competant test system and with experience testing tweaks and especially controversial tweaks. (You wouldn't want expectation bias to enter the test, now would you? LOL We all know that's not a very scientific approach.) Isn't Investigation really the most important part of skepticism?

Quote:

Having finally visited the website devoted to “Brilliant Pebbles” (BPs), I can now apply the general principles discussed earlier and make some concrete comments. Of the issues arising in the paper, I will focus on one. I am not a merchant and have nothing to sell. I merely wish to offer an example of the sort of scrutiny and criticism to which I have been subject in my world.

I can harldy wait.

Quote:

Suppose a sugar pill with blue dye costing $100 per pill benefitted 20 percent of patients, and that a sugar pill with red dye costing 2 cents per pill similarly benefitted 20 percent of patients. The claim that the blue dye was the cause of the effect would not be supported. The effect of the blue pill is fully explained by the simple fact that it is a pill. That is, it is fully accounted for by its “pillness.” It is only by exceeding this threshold that something would deserve to be called a drug. Also, the difference in price is not justified by a difference if performance. Failure to outperform a placebo earns a candidate drug no medals. If an expensive treatment works, but an inexpensive placebo works just as well, it allows patients to enjoy the effect while simultaneously saving money.

You are making a strawman argument as your assumption, or at least untested assumption, that "an inexpensive placebo works just as well" is false, another illogical argument, again right out of Zen and the Art of Debunkery. I'm starting to detect a pattern here.

Quote:

The principle issue regarding BPs is that the discussion, as presented on the visited website, offers an explanation for why they would outperform generic, amorphous stones, but no report of the use of the control that would isolate this factor.

Actually, I do no such thing. I don't offer an explanation why the BP outperforms other generic, amorphous stones. I do, however, explain how BP works. If you wish to hire a theird party tester to test amporphous rocks vs BP free free. In fact I wish you would.

Quote:

The difference in performance between BPs and generic stones is either statistically significant or it is not. If it is, then the null hypothesis is falsified and mere “stoneness” or ”petrosity” is shown to be insufficient to account for the observed effect. If the difference is not significant, then the null hypothesis stands unfalsified and the effect of BPs is explained by their “stoneness,” which is also a property of generic stones that are available in vacant lots at a substantial discount.

I have not addressed generic stones. That is yet another fallacious argument on your part. The fact that some stones are found in vacant lots has no bearing on my explanation of BP, anymore than any other type of material has any bearing. You might as well ask why didn;t I compare BP to ALL other types of material? LOL

Quote:

Which is the case? The answer may be published elsewhere, but there it is no report of it on this particular website. In the absence of this control, there exists only an explanation in search of an explanandum. Thus, this report taken in isolation offers little of interest to science. (Its value as poetry is left to the appraisal poets.) This is not to say that BPs do not work. But even if they do, this is really only worth knowing when it is also known that ordinary stones do not. A theory is offered from which is derived the expectation that ordinary rocks would not work, but this remains an untested hypothesis, not empirical data. Woulda-coulda-shoulda is not a valid category in logic any more than it is in Simon Says. This indeterminacy leaves the null hypothesis on the throne, yawning and trimming its fingernails. Ludwig Wittgenstein reformulated Ockham’s Razor to say that whatever is explanatorily unnecessary is meaningless. A defense attorney should have to fix what ain’t broke only when the jury is too stupid to recognize when it ain’t broke. Competent jurors and scientists need no such help. Exerting effort to get to where one already sits (fait accompli) is, as per Wittgenstein, unnecessary, wastefully and meaningless.

All of this discussion and philosophizing with regard to ordinary rocks vs mineral crystals is irrelevant, as is comparison of BP to other types of materials. Out of scope.

Quote:

This should mitigate possible concerns about veracity. Even if this BP business is all a lie, it is an insufficient one, such that even if it is all true, it is still not enough. It takes the form of an accent fallacy, as if one said, “Jim Austin was sober today.” Perhaps he always is. “These stones work” may be a true statement, but it raises the question: “Which ones do not?” Crystaline structure per se can be found in table salt, a bag of which should be tried before resorting to anything more expensive in case its crystals turn out to be fortuitously tuned for this application.

It might all be a lie or a hoax. But how would you know? You have not investigated it, only relied on your skeptic's arguments, which, as I've pointed out, are often fallacious and/or illogical.

Metals such as copper and silver have crystal structures as well, but I'm not suggesting using metals as a substitute for BP. Nor do I suggest on my web site that all mineral cystals have the same effects in audio applications. Quite the contrary - I explain that differnt crystals have different characteristics and effects.

Quote:

Health insurance providers do not pay for experimental treatments because they are waiting for the other shoe to drop. In science, ignorance of the existence of that other shoe is intolerable. Simply being “pretty rocks” is not grounds for derision, but if confession were to reveal this to be a hoax, grounds for surprise would be lacking.

You certainly wouldn't be the first self-proclaimed skeptic to come along and react to the "pretty rocks in a jar." or to use words like hoax and ignorance. BP has been "reacted to" for 8 years by many skeptics in their rush to judgement. But one must ask, whatever happened to scientific curiosity and the scientific method?

Quote:

Little controversy arises about all this on the superficial level of commerce: a commodity is produced, people pay money for it, end of story. Merchants are not always required to show their work exhaustively and science is not always the issue. In a recent documentary, when astronaut Frank Borman was asked about the science objectives of the Apollo 8 mission, he replied, “Science?!” Additionally, the occupancy of hot kitchens (such as peer-reviewed scholarship) is voluntary. (“Doc, it hurts when I do this.”) However, when science is the issue, incomplete experimentation and testimonials submitted by persons of uncertain ontological status are not enough to command scholarly attention.

Skeptics are free to investigate any claims they feel are outrageous, and any scientific group is free to test those claims. So far, I have heard the same old skeptic arguments, the ones straight out of Zen and the Art of Debunkery. By the way, you left out one of the skeptics' favorite arguments - so I'm going to help you out here - that argument is: why are audiophiles who believe in Brilliant Pebbles often the same ones who believe in homeopathy and UFOs? LOL

Quote:

With regard to the Jorgensen exchange: The null hypothesis is not an opinion, and was in force prior to the cited Lancet paper. The report of no effect does not change the situation from ignorance to enlightenment. It reports the situation to be unchanged. Upon acquittal, the status of a criminal defendant does not change to innocence from something else. Jorgensen’s students need to understand these things if they hope to qualify, respectively, for science or jury duty.

Whatever one should or should not do prior to forming an opinion, my students would not be taught to form opinions but to recognize facts.

Apparently haven't even followed your own advice as most of the philosophizing and objections in your post resulted from having formed a foregone conclusion based on opinion, not facts.

Cheerio,

Geoff Kait
Machina Dynamica

Brucest's picture

I apologize for joining this discussion late.  I am a recently returned subscriber, and unfortunately I find much of the same mindset that caused me to let my previous subscription lapse.

This article is an exception, and I find it one of the more interesting comments of the objectivist vs subjectivist debate in audio.

I take the point about the ultimate insturment for judging equipment to mean that you and I might listen to the same equipment produced sound, and disagree as to whether A is better than B.

However, it seems to me the critical question is whether the ear is consistent. 

For example, one listener, hearing both a solid state amp and a comparable tube amp, might prefer the tube amp on the grounds that the resulting sound is sweeter and warmer.  This preference can be very scietifically investigated if the listener is consistent in forming his preferences.

For example, one could ask as to what are the differences between the amps from various technical measurements.  We are likely to observe a rather large difference between the two amps with regard to the most basic measurement, HD.  Posit that the tube amp has greater HD.  

A logical question: is the preference related to the greater HD?  The test: by some means, introduce an equal HD to the less expensive solid state amp's output.  If the preference disappears, then we can conclude it was associated with the higher HD.

If this is true of a large number of persons interested in sound reproduction, then we have an interesting result.  Such persons should not be expected to change their preferences simply because they seem to be based on an "imperfection".

Rather, a relevant question is whether the preferred HD might be more economically and controlably introduced in the solid state amp.

A seperate question is which reproduction is more "accurate".  This question is potentially of less interest, because it's irrelevant to what one might prefer.  However, it still may be useful as a guideline for examining preferences.

It seems to me this question, with some difficulty, could also be investigated scientifically using the ear as the instrument.

Crudely one might imagine the following experiment: in a concert hall with a quartet, a person is equipped with a set of very good headpones.  Stereo mikes colocated with the listener, capture the sound of the orchestra and the signal is routed through a solid state and tube setup alternately.  The listener would then be able to hear the the sound without the headphones, and then (blindly) be asked which amp more closely reproduces the sound.

There are a number of possible outcomes.  The listener may find one or the other of these to most closely capture the sound.  The listener may find them both so different from the natural sound so as not to be able to make a conclusion.  He may also find the less accurate sound to be preferred to the original.

IMO, a substantial problem with the subjectivist approach is that little guidance is provided as to technical design criteria.  Is XYZ multibuck peice of equipment sound "better" because of a design element whose effect can be objectively correlated with the subjective preference?  

If this is the case, then engineers can pursue less costly means of achieving that design criteria.

In the case of the previous example, the technical solution might be a solid state amp, much less expensive than tube, which can introduce a range of desired HD.

X