You aren't being scientific. You keep repeating the same mantra over and over. "How do they know if they don't do a blind test?" As I've already said more times than I can count a blind test doesn't prove anything. Do you honestly believe that if a blind test shows there are no audible differences between two components that means that the two components always sound the same to all listeners in all systems in all rooms? Do you actually believe that if a controversial tweak, let's take Mpingo discs for example, is shown to not make an audible difference in a blind test that proves that the Moingo disc is a fraud and doesn't work? Do you realize how dogmatic and closed minded that sounds?

Geoff Kait
Machina Dynamics

What is the purpose of this debate?

It sounds like the OP has lost faith in at least a couple of audio expert reviewers here on Stereophile.

It further sounds like the OP is suggesting Stereophile introduce more formal testing procedures between pieces of equipment.

I have no fundamental issues with anything spoken. But do have some observations and concern with HOW the OP's opinion is being expressed:

1. Stereophile has in fact produced volumes and volumes of detailed test results..and drops signal analysis and other terminology like Fourier analysis left and right. So perhaps the OP is perturbed about a specific review? If so, then just be specific and address your precise concern(s) with a precise review. That way this will be much more productive; without getting personal and destructive.

2. Experts are just that: experts. They have gotten there by a combination of education, experience, and their own repeated constant ability to produce successful results to their customers and colleagues. Last time I looked, this is not "american Idol" and we are not here to "vote off" experts. Good grief. Relax, old sport.

3. This is my most concerned reason for even typing. It kind of upsets me on a visceral level to suggest that a staff member of a well regarded magazine be tested for hearing and their results published. That sounds like a page out of "Mein Kampf" or Karl Marx. The sheer notion of that easily leads to the *very* slippery slope of invasion of privacy and other Bill of Rights Violations. This is America. Please just let the man do his job. If his superiors feel he is medically unfit, they will address it. Flaming the poor gentleman here is definitely NOT the solution; though I certainly will not get in the way of your own "Freedom of Speech". But with that freedom comes a responsibility: to use at least some decorum and kindness and be a little more respectful of the people on staff at this long-lived and well regarded magazine.

Regards,

Ronald R. Stesiak PhD (h.c.)
National Science Foundation

DISCLAIMER: I am not a staff member nor do I have any affiliation with Stereophile or any parent organizations or subsidiaries thereof. I'm just a concerned reader.

mosfet50:

I fear your style of debate has merely fallen to being rude to a friend I have made here on these forums. Would you please just explain your point in detail so that your freedom of speech may be respected and addressed and we all can move on?

Regards,

Ron

You wrote,

"What's a blind test? It's a test where an individual listens to two or more pieces of equipment and repeatedly differentiates between them or finds no difference. So, if there's no audible difference how can he hear a difference??? Think about what you're saying. Did I say another individual couldn't hear a difference? Never! The blind test is singular to the specific individual. Period."

I knew we'd finally agree on something. You are agreeing with my statement finally that two observers might have different results or draw different conclusions if they perform the tests independently. This is what I've been saying all along. But the catch is that you can't have your cake and eat it, too. Why would you say listener A is correct but listener B is wrong? They are both simply data points. Now, if you were to make the statement that YOU had such and such results when YOU performed a test of some sort then I would probably pay more attention to your arguments and give you the benefit of the doubt that you were reportIing what you heard. I don't think anyone is really saying that NOBODY IS INFALLIBLE. One the other hand isn't it logical to assume that those who practice the art of listening and reviewing are less prone to error? Everything is relative, so even reviewers can be wrong sometimes. Would you say reviewers in general are MORE prone to error than the average person?

To err is human. To forgive Devine. Lol

Geoff Kait
Machina Dynamica

You will see I gave a specific example of a device (the Zanden DAC) that Michael Fremer considered superlative and John Atkinson found to have over 20% THD. Now don't get me wrong, I know people who listen to SET amps with close to 10% THD but they admit that what they hear is far from what is on the media: they just like it this way.
However, I would expect an expert to identify 20% THD and call it as such, no matter if he likes the sound - after all, the potential buyer might prefer better fidelity instead of euphonic distortion.

As for DBT, it's useless! I have a track record of over 30 years in "audiophilia nervosa" but a few years ago I got fooled by listening conditions. Let me explain: at that time I listened mostly to electronic music, for reasons I don't feel I have to make public. I asked a dealer (and friend) to bring some new interconnect to better complement my system. Well, he came with two models from two brands, I listened to each of them (so the test was not blind) and I decided to keep the cheapest-looking model. BTW, price was not an issue since any of them was free as part of a system update. Well now, I lived happily with the new interconnect until I started to listen (again) to a more diverse kind of music... that's when I cringed!

And a short notice for mosfet50: give me two "reasonable" speaker cables (I mean, not a top of the line designer cable and some romex wire) and I guarantee that I can set up a system and select the proper music to make any of those cables sound better than the other.

I asked, what do you do when two independent tests result in two opposite results? One test results in a clear audible difference, the other in no audible difference. Do you throw them both out and start over? Do you throw out the one you don't agree with?

Cheers,

Geoff Kait
Machina Dramatica

mosfet50 wrote:

"And a short notice for mosfet50: give me two "reasonable" speaker cables (I mean, not a top of the line designer cable and some romex wire) and I guarantee that I can set up a system and select the proper music to make any of those cables sound better than the other."

I'm sure you can but it wouldn't be a double blind test!! What's a DBT? ALL THINGS BEING EQUAL!!

Well yes, all things would be equal: same system, same room, same SPL, same... well, EVERYTHING. I would not change anything, I'll just select the best system and music to showcast the performance of cable one vs. cable two.
Still don't get it? Well think about a DBT involving a SET and a SS amp. Most likely the SET won't be able to offer anything close to the dynamic range of the SS amp but it will (probably) be richer in harmonic content. Now let's suppose I organize a DBT with Bach's violin sonatas, which amp do you think will get the best results? OK, now change the test music to Mahler...
See what I mean? And yes, the test can be DB but the organizer still gets to choose the music.

I was referring to blind tests. But as I have been saying all along it could be ANY kind of test. That's kind of my whole point. Blind tests are not some magic thing that proves something works or doesn't work. Or that component A is better than component B. And the test could be between two different components or the test could be testing a single tweak or device to see if it was audible.

From an earlier post,

""What's a blind test? It's a test where an individual listens to two or more pieces of equipment and repeatedly differentiates between them or finds no difference. So, if there's no audible difference how can he hear a difference??? Think about what you're saying. Did I say another individual couldn't hear a difference? Never! The blind test is singular to the specific individual. Period."

I knew we'd finally agree on something. You are agreeing with my statement finally that two observers might have different results or draw different conclusions if they perform the tests independently. This is what I've been saying all along. But the catch is that you can't have your cake and eat it, too. Why would you say listener A is correct but listener B is wrong? They are both simply data points.

mosfet50 wrote:

"I knew we'd finally agree on something. You are agreeing with my statement finally that two observers might have different results or draw different conclusions if they perform the tests independently. This is what I've been saying all along. But the catch is that you can't have your cake and eat it, too. Why would you say listener A is correct but listener B is wrong? They are both simply data points."

Here's an example. Suppose person A performs a blind test on two amplifiers and find there is no audible difference between them. So, you would probably say then, well that proves that the two amplifiers under test are essentially the same sound wise, right? But what if they test person A's hearing later say the next day and find out his hearing is limited to 12 KHz frequency response. Then what? Do you disqualify the test? What?

I never said one listener is correct and the other isn't. Go back and read what I said. It's about what one specific individual hears, the test is singular to the individual. It has nothing to do with what another individual hears.

It's not a correct or incorrect condition. Listener 'A' is attempting to discern if he or she can hear a difference in two pieces of equipment. What listener 'B' hears or doesn't hear is a non sequitur!

Rob

OK, let's look at this from a different angle. You said just now that it's "about what one individual hears and the test is singular to the individual." I couldn't agree more. Of course each test is specific to the listener AND the test system. But what if that particular individual can't hear as well as the next guy for some reason or another or doesn't have the SKILL that the next guy possesses? You cannot always find ideal listeners who know what to listen FOR and whose physical hearing is sufficient for the task. What of test subject's system is mediocre or has a mistake in it? Then what? If I'm not mistaken you seem to be of the opinion all listeners are the same and all audio systems are up to the task. f tests were as straightforward as you would have us believe then there would not be such a long debate on this very subject. Especially blind tests.

Geoff Kait
Machina Dynamica

mosfet50 wrote:

Wrong again!
The listener only has to satisfy his or her own ears. If he or she can't find a difference between two pieces of equipment then it ends there. They're auditioning equipment for themselves, no one else! Again, what someone else hears in the same test is a non sequitur.

All listeners are decidedly NOT the same. I never said they were the same or all systems were the same.
Rob

Have you ever heard the scientific method axiom, "the test must be repeatable?" Well, how can the test be repeatable if for any number of reasons it was not performed correctly? That was actually the reason that the famous botched experiment got so much notoriety. What was it? Oh, yeah, the cold fusion experiment!
If the person is hearing challenged how can he trust his ears? And if he is ALL THUMBS like a lot of folks are (unfortunately) how can he ever trust himself to set his test system up properly? Many systems are out of absolute polarity and their owners don't know it. Their owners don't even know what absolute polarity is. Answers at 11.

Geoff Kait
Machina Dyabolical

Nice chatting about this. I suspect we have reached the end of this particular debate.

Cheers,

Geoff Kait
Machina Dynamica

First off, I haven't read every word. What I did read got me back to the topic, in my mind anyway, that we need a high end audio testing lab. One that allows us to do both short term and long term studies and testing of all the issues, methods and practical applications involved with this industry. This way, even though many will disagree with results, there will be a common open ended place of study that is able to accommodate as many of the ideas as the user wishes to invest in.

The Institute of High End Audio.

I vote for Las Vegas as the sites location. Maybe others could give their opinions on placement and potential activities. I recommend Las Vegas, for a few reasons. One I'm here, two it would be at the home of the CES, three the desert conditions, four I've done this before and five travel accommodations.

thoughts?

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

I think this hobby has opened itself up to "In My Opinion" so blindly that it has created a monster that doesn't need to exist. I agree, it truly has become the world of "Hype". As a result, an industry that should be talked about on every front page has turned itself into a dramatically over priced graveyard. The price-tags replaced common sense, and the sad thing is the number of people who buy into it.

The industry has become so out of balance that the designers themselves will tell you behind closed doors that the "price-tag" is a marketing tool. "you have the biggest product, that cost the most, and you will be considered".

You can't exist forever as an industry that is half engineer half artist without having an equal playing field. The loudest voice doesn't win, and the most expensive is a scam if your talking about "music". It's not a scam if your talking about jewelry, and many audiophiles have completely put on their ear blinders when it comes to how great sound is made.

There must be a place where "IMO" is put to the test and put to the test in every way needed to present "proof & truth". An open opportunity for those who make claims to present their skills and talents. Not a negative place but one that is positive. A place that can explore as many tests as is needed or wanted for each person. I think it should also be a place that has it's own library, and discussion forum, so all can contribute to the knowledge pool.

There are some that will look at specs (and make specs) only and others listening only. Those that base their listening on theory and others that lean toward doing. But a place where all can exist in the spirit of the industry and hobby, and allow equality to all the members who are willing to be put "to the test".

I'd personally like to start talking about such a place, as well the possibility as to what I could give to it.

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

But, I can't imagine a valid way to make this even possible. I'd love to be able to buy an audio component that I knew would meet my tastes simply by some sort of reliable measurement that would fully convey what the component sounds like to me and my ears, system and room.

This seems more like the usual "let's test the testers" kind of thing from the same sort of people that have never understood what Stereophile and their reviews are meant to do and in what way they are of use to potential consumers. Reviewers aren't telling you what to like, they are telling you what the component sounds like in the context of their systems, having listened to it over a lengthy period of time and in comparison to their reference system and their accumulated experience of listening. That's pretty much it. Of course, each writer does this in their own style of writing and some are more thorough than others, but that's still pretty much all they are saying.

I don't think you guys have really thought this through. Do you have any idea how expensive and time consuming this would be? And that's with the realization that the results can't possibly be definitive without an expectation of knowing what to measure that correlates to what something will sound like. AES can't even achieve that goal and they have decades of development from the brightest minds in the business sharing their designs, data and thoughts on every level of audio.

This sort of thing goes with the territory when art and science becomes intermingled. But, it's not possible to remove one from the other and subjectivity is at the heart of all artistic expressions.

If somebody knows what needs to be measured that isn't being measured, say what it is. So far, we have Mikey's and Art's hearing. Anything else?

You wrote: "Reviewers aren't telling you what to like, they are telling you what the component sounds like in the context of their systems".
Well no, reviewers are just telling you what they think the component sounds like in the context of their systems, so it's important to know if they are able to identify the real sound they get.
As I wrote before (twice!) when a reviewer is fascinated by the sound of a component with over 20% THD I am entitled to ask myself whether:
a) he cannot hear?
b) his system is broken?
c) his preferences regarding sound are, well, "strange"?
d) he (or the magazine) got a perk from the manufacturer?
So this is not about their writing style, it's all about their listening ability (not questioning their honesty, of course)!

There are numerous speculative conclusions that could be drawn. I might draw the conclusion that Mikey is so engrossed in analog sound that he might not be the best review guy to be listening to SOTA digital. But, that's speculation. I might even speculate that he did a lazy review or maybe ask myself if it's possible that the particular distortion characteristics are all that damaging to the final sound.

In the end, I might say to myself that given MF's decades of experience, that's just puzzling and leave it at that.

It's hard to draw any conclusion off of something like this. But, if you respect JA's role as the Editor, you might just snicker a little at the whole thing. The fact that it was published is far more telling to me than the fact that it was an oddity.

I want to say though, those times I had these places were some of the best times I have ever experienced in my life. Ten thousand feet of do anything you want audio playland. Not that I would want to do anything that size again, but I think a full size house given to the project would be manageable.

Question is, if there was a place, would the people who have claims to stake come and be tested, and be a part of developing some tests?

One test I'm thinking of is the one I mentioned back around the CES time, where you can put on clothing that monitors your responses to what you are hearing. This could be a next level to the A/B test. I'm not talking just an old school place here, but something that offers a fresh approach to how we measure what we like.

I think it would be very cool to put innovation to work.

The other part to what Catch says will work itself out. I'm not sure we have recovered from the loss of the mom and pop stereo/stores, but the stores we now have, have been taking big steps in in-home reviewing. That I think will only get better.

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

And I certainly appreciate the need to use sound engineering in designing components. But, things can still measure the same and sound differently if what is being measured isn't what is equating to a difference in sound. That doesn't mean that a bias is influencing perception, necessarily, but rather that not everything that is being measured makes an audible difference or that what makes a difference is being measured.

Cables, for example, from an engineer's perspective, should have no effect on the sound if it meets the design requirements for a given purpose. The fact that they do sound differently suggests that what an engineer is testing for in a given cable doesn't measure everything that is audible. If the engineer simply dismisses this as bias or placebo rather than explore what they aren't testing for then they are using their own bias to dismiss something rather than discover why. BTW, Art has only recently accepted that cables do sound different. He did not go down willingly. Ha!

I'm really, and honestly, not averse to testing and I actually do appreciate the stubborn nature of science. But, when it comes to having the requisite curiosity to ask "why" and want to explore the answers, I'm happy to sit in the subjective camp and tell the nay sayers, "You go figure out the why, I don't need to know why to know what I hear."

The problem is that scientific testing would affect the living of "golden ears" reviewers and their opportunity to get expensive gear at (ahem!) "accommodation" prices and also the sales of less-than-expertly-designed stuff that mediocre engineers foist on unsuspecting buyers.
To quote from the movie "A Few Good Men": You want the truth? You can't handle the truth!

Yes, it's true well not in the words you use, but it's true that many audiophiles like tubes as opposed to solid state but not because they like "tube distortion" but because the distortion of tubes is less irritating or objectionable and actually much better when you look at the harmonic structure of tube amps. That's the real reason tubes by and large sound better than many solid state amps, which IMHO have a well synthetic and metallic and unnatural sonic signature. Yes, I know, there are some good sounding SS amps. I'm generalizing. This is definitely a case where science is not as reliable as our hearing. That THD nonsense was the biggest scam out there since 90 dB dynamic range for digital and the importance of slew rate and flat frequency response.

Geoff Kait
Machina Dynamica

"There are many forms of distortion." - Geoff Kait

Geoff Kait
Machina Dynamica

mosfet50 wrote:

That's called 'stating the obvious' Geoff.

Rob

I know of distortions you never ever heard of, trust me. Stating the obvious? I don't think so. But a question -why does the statement you quoted from Tung Sol mention "when the amp is overdriven,...." Who overdrives their amps? I'm not referring to distortion resulting from overdriving or clipping. Am I missing something?

Geoff Kait
Machina Dynamica

That is typical of pseudo skeptics who claim blind tests solve all of audiophiles woes, including expensive cables, controversial tweaks like the Mpingo disc, Shakti Stone, coloring CDs, demagnetizing CDs, ionizing CDs, CD enhancing fluids! sending CDs or anything off to the cryo lab, freezing CDs or records in the home freezer, the Intelligent chip, SteinMusic Harmonizer, Silver Rainbow Foil, expensive capacitors, tiny little bowl resonators, The problem with this sort of retarded thinking is that you can find it all over the Internet and from all sorts of so-called experts. Do you know the definition of an expert, Rob? An expert iis someone who used to be a drip under pressure. Get it? Ex-spurt. An expert is also some guy in a cheap suit 50 miles from home. ;-)

The further irony of course is that the same pseudo skeptics that maintain blind tests will settle all controversies are the same ones that actually don't do the tests themselves, one supposes because their mind is already made up on all the audio controversies such as cables, which just might be the longest running one, you know, having started around 35 years ago. And making ones mind without tests, without so much as a thank, you, m'a'm is just SO unscientific, don't you think? For anything, even for lithium vs alkaline batteries.

Geoff Kait
Machina Dramatica

The esteemed MIT professor was giving his opinion, too. Just like you are. None of us are giving any proof. Not even evidence. Not even the MIT professor, who has the audacity to say audiophiles like the "pleasant sound" of some high end cables. Strictly wishful thinking of the highest caliber. Well, la dee da! You don't have to look too far for someone with great credentials to poo pooh high end cables or high end anything for that matter. I'd wager AES is probably in agreement with your professor's view that high end cables are snake oil. The professor even weasel worded his view trying to back-peddle when he opined, "and any sonic differences that might exist can be corroborated by measurements." That's priceless! The MIT dude talks like you do, in generalities and more or less from a position of pomposity. I also wager he doesn't even have an audio system. Just a guess, but I suspect he's never actually measured any cables either. That would be too much trouble. Measurements are for those little people, the ones down THERE.. ;-). Ooops, I guess I'm not quite out of this thread yet. ;-)

By the way citing some old MIT dude on the subject of audio is actually a logical fallacy! it's called the appeal to authority. It's a mistake in Logic.

Geoff Kait
Machina Dynamica

Before measuring anything I check if the measurement chain is calibrated. Sometimes it is, some other times it ain't, so I need to do some calibration work before starting the actual mesuring process. The problem is calibration is not always possible so I have to make do with a systematic error... but this doesn't hurt if I know the influence of said error (say, a 0.01 mV DC bias) - I can substract it from the results before analysis. What I certainly don't do is throw away the measuring gear which might otherwise perform flawlessly.
Well, the same goes for reviewers: I don't suggest those with less than perfect hearing or obvious biases should be dismissed, I only expect these facts be published so the readers are given the opportunity to apply a "systematic correction" if they feel the need to.

I was trying to say YOU were guilty of a logical fallacy, the Appeal to Authority, not that the professor was guilty although he actually probably is too. By using the good MIT professor to try to prove your point you were using an Appeal to Authority. Besides anyone can use the WORD science in a sentence. It doesn't necessarily mean the sentence is scientifically correct. It's another logical fallacy. If I say I have a scientific background, which is actually true, therefore whatever physics or electronics or biology statement I make is automatically TRUE that would be a logical fallacy, right? If Appeal to Authority were an acceptable form of debate all anyone would have to do to win and argument no matter what it is is Google the Internet until you find some MIT professor or Harvard professor or PhD who backs up your argument, as I already pointed out. You DO realize, don't you, that scientists do not agree with each other on scientific arguments. Not all scientists share your enthusiasm for blind tests, for example. You didn't think this was going to be easy, did you? Lol

Geoff Kait
Machina Dynamica
Run silent run deep