Scientists don't agree on much of anything unless it's a law. Theories frequently have competing theories, that means scientists don't agree. Didn't you ever notice how long it takes for anything new to get accepted by the Scientific Community? That's why you see a lot of old geezers accepting the Nobel Prize, it's often twenty or thirty years after their discovery. A little slow on the uptake.

Geoff Kait
Machina Dynamica
Advanced Audio Concepts

Let's look at it from a different angle. Suppose I say that I've done blind tests on one of my more controversial products, let's say it's the Clever Little Clock, the battery powered clock that you place somewhere in the room. And I publish the results of those blind tests, which are very favorable, on my web site. Now, do you really think everyine is going to jump on board and say, wow, the Clever Litlte Clocks really do work and that customers will be lining up in droves to buy them? No, probably not, right? And why not.? Well, for one thing they might not believe me. Or they might perceive the results as just a marketing ploy. Just as you're skeptical of reviewers' claims about listening tests, folks would be skeptical of manufacturers who publish test results and tend to pick at them like ducks on a June bug. You suspect them of something, maybe faulty hearing, but you have no real proof. You going on gut feeling and that's not very scientific. You kniw, real skeptics can never be convinced of these things. There is no evidence powerful enough to sway them from theiir beliefs. They would have to turn in their Skepticsl Society membership card. Just like you most likely cannot be convinced by ANY test results that lithium batteries sound different from Alkaline batteries. Right?

Geoff Kait
Machina Dynamica

Because in a battery-operated device there (usually) are no storage capacitors so the dynamic might depend solely on the instantaneous discharge capacity of the batteries.
BTW, there are batteries recommended for high-drain, short use applications (think flashlights) and others recommended for constant-drain, long term use applications (think smoke detectors).

However, batteries cannot sound different if they are OUTSIDE the signal path, i.e. in an optically-coupled volume selector.

Listening experience is crucial in evaluating audio. Experience, in general, is valued. What you refer to as "Golden Ears" in a dismissive manner is akin to dismissing a wealth of experience from which to compare, contrast and evaluate. Just because we can all hear, more or less, doesn't make us all equal in discerning the things that encompass high fidelity.

If the criteria for being qualified to render an expert opinion on audio was the ability to hear all the way out to 20K, then the only experts would be 18 year olds without any experience or knowledge on the whole topic to be qualified to even have an informed opinion.

Nobody is oblivious to the imperfect nature of subjective reviews, hell, this goes on in all sorts of fields.

It's healthy to be skeptical, but it's better to be skeptical with a bit of humility in the presense of people with so much more experience and knowledge on a subject than you have when their accumulated evidence differs from yours. And, no, I'm not talking about all us out here in the peanut gallery, but rather all the members and contributors to the Audio Engineering Society. Which, btw, would give you a lot more to think about if you should be curious enough to join.

Maybe you can be the guy who stood on the shoulders of giants and discovered a predictable way to measure all the multitude of things that impact what we perceive as realistic sound. Seriously, I think you would find AES fascinating and inspiring.

It's actually not a valid argument to say many people believe DBT is a valid system of evaluating equipmentt therefore anyone who doesn't use DBTs is unscientific or that their opinion doesn't count. That's some sort of logical fallacy, give me a day or two and I'll get back to which logical fallacy that is. For every person who thinks DBTs are valid there is a person (at least) who thinks they aren't valid. Furthermore, whatever test protocol you can come up with for performing DBTs scientifically there are many who will tell you your test protocol is flawed. So you can t win.

Another thing - you keep saying over and over again that these reviews couldn't pass a DBT. But how do you know? Have they ever submitted to a DBT? Or are you just assuming they wouldn't be able to pass one? One more thing, you criticize senior reviewers for their hearing quite a bit. How do you know WHAT their actual hearing capabilities are. Or are you just assuming? Speaking of which, I bet your hearing is not all you're making it out to be. Have you been tested recently? I got a cup a coffee that says you haven't.

Okay, here it is. Didn't take me very long, either.

Argumentum ad populum
EditWatch this page
"Ad populum" redirects here. For the Catholic liturgical term, see Versus populum.
In argumentation theory, an argumentum ad populum (Latin for "appeal to the people") is a fallacious argument that concludes that a proposition is true because many or most people believe it: "If many believe so, it is so."

This type of argument is known by several names,[1] including appeal to the masses, appeal to belief, appeal to the majority, appeal to democracy, appeal to popularity, argument by consensus, consensus fallacy, authority of the many, and bandwagon fallacy (also known as a vox populi),[2] and in Latin as argumentum ad numerum ("appeal to the number"), and consensus gentium ("agreement of the clans"). It is also the basis of a number of social phenomena, including communal reinforcement and the bandwagon effect. The Chinese proverb "three men make a tiger" concerns the same idea.

Cheerios,

Geoff Kait
Machina Dynamica

This thread serves as a good example of talking without doing. I run into this alot on this forum, always asking myself why.

I understand the talk, but if your not "doing" things together in a practical application setting, it's "only" talk. As far as science, testing and application go hand and hand and is a requirement for moving from theory to true science.

I might have missed this, but do any of you belong to "science forums"? Do any of you belong to "testing forums"? Do any of you hang out with "practicing scientist" or have access to "science labs"?

From what I have seen since I've been posting here are people (mostly men) talking science but when you get to know them they aren't actively involved in the scientific labs and programs they are refering to. If you don't start talking about the doing instead of your particular belief on what is and isn't, at best you are still at least one step away from proving anything. I mean what power does one male ego have over another without the actual competition? I'm not going to go to an armwresting contest and talk my way to the trophy. With something as variable as music and music reproduction, I'm ceratinly not going to get anywhere without the "doing" on either end of the experience to testing without the actual "doing". You guys might as well measure your biceps and be done with it, because even though this is titled "Opinions and Science", all that has been going on is "opinion". Opinions about science, is far from science. And those who have those opinions are far from scientist without the "doing".

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

Proving once again there is no Stereophile thread that cannot be converted into a pulpit for the Tuning generation at the drop of a hat. You don't even have to address the topic of the thread, just a little hand waving will suffice. Just pull the good old reliable walk the walk routine and And voila - a Tuning bulletin board is born. Reference this - The OP mentions three things missing from reviewers:

1. The hearing spectrum.
2. The double blind test.
3. The AB subtractive test.

Geoff Kait
Machina Dynamica

Geoff, I understand your going to put up stupid pics on these threads, but might I ask that around Memorial Day that you don't disrespect the troops or their days of recognition, on this thread and on the Memorial Day one! If your going to use pics and crayons to proove your right to express yourself, it would be good to do this with some decency, respect for those fallin and a little taste.

If you could at least do your trolling with some sort of respect when it is due, I think it would go a little further.

I'm sure you weren't thinking about this at the time of posting, but this is a sensitive time for many of us.

thank you

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

michael green wrote:

Geoff, I understand your going to put up stupid pics on these threads, but might I ask that around Memorial Day that you don't disrespect the troops or their days of recognition, on this thread and on the Memorial Day one! If your going to use pics and crayons to proove your right to express yourself, it would be good to do this with some decency, respect for those fallin and a little taste.

If you could at least do your trolling with some sort of respect when it is due, I think it would go a little further.

I'm sure you weren't thinking about this at the time of posting, but this is a sensitive time for many of us.

thank you

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

It's always a good time to show what we were fighting against. My father fought in Anzio and North Africa. I'm assuming my post went over your head. Or at least you're pretending it did which is certainly understandable. any more pearls of wisdom?

Geoff Kait
Machina Dynamica.

I can see someone developing a test where the listener has their brain activity monitored as they listen to establish whether a difference is being perceived by the brain, independent of whether the listener actually perceives any difference in sound.

I see it as a way to get around not being able to measure everything. Or, as Donald Rumsfeld might say, the known knowns, the known unknowns and the unknown unknowns. ,)

geoffkait wrote:

One more thing, you criticize senior reviewers for their hearing quite a bit. How do you know WHAT their actual hearing capabilities are. Or are you just assuming? Speaking of which, I bet your hearing is not all you're making it out to be. Have you been tested recently? I got a cup a coffee that says you haven't.

1. Well this is exactly the point: we cannot know WHAT their actual hearing capabilities are, we just have to assume (spare me the idiot joke with Uma Thurman). If the reviewers' hearing is good why don't they post recent audiometric tests?
2. mosfet50 does not have to post any hearing test since he's not asking to review audio gear. But when you ARE reviewing audio gear and expect to be credible, well, publishing your hearing test results is mandatory.

Now you get it, geoffy-boy?

thing, you criticize senior reviewers for their hearing quite a bit. How do you know WHAT their actual hearing capabilities are. Or are you just assuming? Speaking of which, I bet your hearing is not all you're making it out to be. Have you been tested recently? I got a cup a coffee that says you haven't.[/quote]

1. Well this is exactly the point: we cannot know WHAT their actual hearing capabilities are, we just have to assume (spare me the idiot joke with Uma Thurman). If the reviewers' hearing is good why don't they post recent audiometric tests?
2. mosfet50 does not have to post any hearing test since he's not asking to review audio gear. But when you ARE reviewing audio gear and expect to be credible, well, publishing your hearing test results is mandatory.

Now you get it, geoffy-boy?[/quote]

Dear Costin, Are you posting drunk again? If you think that listening skill is determined by physical frequency response you need to check back into rehab for a tune up. You might as well demand that reviewers submit the frequency response of the listening room. People's listening ability or skill in evaluating components or cables is not determined by their ears' Frequency Response chart. Since you distrust reviewers as much as the OP why on Earth would you accept whatever hearing chart they submit? What is "Good hearing" according to you? Do you even know? How often do they have to be retested? How often do they have to resubmit a photo? It sounds like according to you all reviewers would have to be twenty year olds. What's the definition of bad hearing. If a reviewer has really good hearing and you have really bad hearing, what possible good would it do to purchase the component or speaker or cable he reviewed since you WOULDNT BE ABLE TO HEAR IT? And if your room has uneven frequency response, etc. How to you expect to hear what someone else hears in an acoustically "good" room? Doh! And finally, are you dumb or just pretending to be dumb?

Try not to get your sensitive bowels in an uproar on this thread, too. Remember, rehab is just a phone call away. ;-)

Geoff Kait
Machina Dynamica

Hi Rob

Do you have a website so I can take a peek at your lab? I'm sorry I missed it earlier (busy couple of weeks).

thanks

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

Catch said

"I can see someone developing a test where the listener has their brain activity monitored as they listen to establish whether a difference is being perceived by the brain, independent of whether the listener actually perceives any difference in sound."

At the CES this year, this was being demo-ed in some of the private think tanks on the main floor (not the audio rooms, but the medical and smart areas). I was surprised how far monitoring has come. I would guess next year this will be even bigger as they now have the monitoring watches and other wearables.

I think the new era of testing is right around the corner.

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

All the things that go into sound perception. Even the brain measurements couldn't possibly be adequately controlled to reveal anything more than "further evidence" as opposed to definitive proof. But, I think most all of us are fascinated to one degree or another with the subject.

Here's a link to a rather long discussion that Robert Harley started some years back. There's the usual static and noise within the multitude of responses, but there are also some real gems tossed in for consideration. Rather than focus on the obvious noise, read it all and you quickly realize the rather daunting task this sort of thing is. But, like lists, (we all love lists!) the comparable worth of opinion mixed with speculation and observations are pretty damn interesting.
http://www.theabsolutesound.com/forums/threads/13/

Unless you team up with the folks who have convinced millions and millions of people around the world that everyone needs to take statin drugs for "high cholesterol" and anti-depressants for all manners of "life problems", you are not going to see this type of testing done any time soon. The data generated by these tests are of little interest and of little meaning to the general public.

Why not devote your time and energy to making the world a healthy and safe place to live?

"People aren't that stupid..."

But I know folks who say that they would never buy a Ford product, even though they've never been in one!

And there are still those millions and millions of people on medications that are not very effective and ultimately don't need.

When a reviewer writes that "component X made his system sound like the live performance" but he is unable to hear anything above 8kHz he simply cannot compare the live performance to his system for frequencies above 8kHz.
Therefore, above that frequency the component under review might sound like a pile of crap and the reviewer wouldn't know this happens!
Oh yeah, the reviewer has good hearing and I have bad hearing. Said who? I just told you I don't need to prove my hearing: I don't review audio gear for a living - but the reviewer does!

Hi Rob

And this is the point I think people are saying.

As you said.

"I'm retired now, one of my hats is audio design, I don't have access to most equipment to test and compare but I do test all my own designs."

A true lab, is extremely hi-tech these days and very expensive. Doing home testing, which is what the reviewers and designers do for the most part is at the hobby level at best. High End Audio is not going to spring for "true testing sites", not going to happen. However the mainstream world has passed up high end when it comes to lab testing. I'm not talking necessarily audio, but a more innovative "smart world", where the labs are extremely advanced. I've consulted for two of these labs, visited the NASA lab and designed 3 others, along with attending the testing courses. I don't know if others here have been to these labs of late, but they are a long cry from A/B and bench testing. Honestly they make high end audio look like heathkits.

Here's what my friends have told me, and I have said this before here. They're not interested in designing for a small camp like the high end audio camp, because the other fields they are involved in are massive in numbers, such as the medical field, when it comes to devices. "Smart technologies" are probably the cutting edge in testing these days, and High End Audio will more than likey adopt some of this technology to use for "body response testing", and the guys here are right. This type of testing will be far more accurate, plus the additive of being personal.

For example: Sam says he hears something, you say it's impossible. At that point a "body response test" could be given to see if Sam did indeed hear what he says he did.

The new age of testing is an individual one as well as variable. True testing is opinion based, but it's also something that can be charted, and will be something that will be design variable to adapt to each listener.

As Chris just said, proof is in the doing. How we measure this proof will be something that inproves more and more as technology has the need.

With this I go back to my earlier comments. How much of this thread is relevant? From my point of view, only the part that is being done.

Reviewers

might I also add this

We wouldn't be having this conversation if it wasn't for the Mike's, John's, Tom's, JGH, Harry and many others who have brought us to this point, in this particular part of the hobby. Will the new generation follow suit? Of course not. "all things become new" and with this passing of the guard will come a different face for high end audio. Different types and styles of the gear, different types of testing, and overall a much wider more accessible hobby. The question is, are we going to look back or forward? Are we going to try to fix old testing or develope the future of testing, opinion, and taste?

I know I have joined this late, but maybe that's a good thing. I read this thread and feel almost like I'm walking backward down the stairs, trying to get higher. I believe some here are trying to shed the old in welcoming the new, and others are trying to stay in the old, puting their fingers in the dike of old school testing. I hope what I'm saying is view with the positive and not the negative.

We are at a very interesting time in this hobby and even industry. There are many things the old school has brought us that is outstanding, but what the future holds, could be something we couldn't have dreamed of, in the way of a hobby becoming more personalized. When you stop and think about it, all of these views are a part of molding and the needed molded future. We can debate the past or build the future.

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

That's a tricky area when you consider that most of the entire world would consider spending $1000 for a pair of speakers as totally insane. Or, $500 for a bottle of wine or...it's endless.

I know people who think I'm crazy for spending 4 figures for a CD player, yet they spend thousands on a few days vacation to the islands or a trip to Vegas.

On the other hand, you can often times tell a lot about people based on their values.

iosiP wrote:

When a reviewer writes that "component X made his system sound like the live performance" but he is unable to hear anything above 8kHz he simply cannot compare the live performance to his system for frequencies above 8kHz.
Therefore, above that frequency the component under review might sound like a pile of crap and the reviewer wouldn't know this happens!
Oh yeah, the reviewer has good hearing and I have bad hearing. Said who? I just told you I don't need to prove my hearing: I don't review audio gear for a living - but the reviewer does!

People say a lot of things, It's a hobby. Get over it. Stop acting like a teenager.

You think reviewers write reviews for a living? Ha ha ha, that's priceless!

I may be slow but I'm ahead of you. Give my regards to your besotted liver.

Geoff Kait
Machina Dynamica

Rob said

"I'm also saying that a good portion of the high end audio market is, well, no higher than a good portion of the middle audio market. For instance, you can spend on a DAC that costs 1k or 10k. They all use a lot of the same chips but one has a hyped rep and the other doesn't but the sound is pretty much the same."

mg

This statement is a big problem for the High End Audio. Not only is it true for the mid-priced but also for the entry level. The units used as parts have come down to numbers made in the production. I have kept friendships going for many years in the parts biz, and the ball game has changed dramatically in the area of bang for buck. In fact the choice I made for my reference CDP was one sold at Walmart for two years before model change. Many scoffed at this until they made the purchase and did the mod. Now these units are bought up as fast as they appear on Ebay or Amazon.

High End Audio, maybe in the 80's and 90's were at the top of the learning curve, but this is no longer the case as the R&D of the big companies have "real" labs for their testing and some super smart designers.

Rob

"Testing for human pleasure response is not an accurate method."

mg

I'm not going to hang in there with you on this one, because I have taken some of these tests and find that the variables (my specialty) is more specific. My approach is not separating the listener/gear/condition/experience as a specific moment into different pieces.

Rob

"Did you ever listen to music one day and love it but didn't the next? We all have and instead of looking at our subjectivity we blame the equipment."

mg

http://tuneland.techno-zone.net/ Tuneland is at the top of this game as we have made audio variably tunable.

Rob

"Sam Tellig might very well have different pleasure responses (although not in a DBT) to different batteries but it proves very little. PLEASURE IS SUBJECTIVE."

mg

Actually Ron we still do test the sound of batteries, and they do (as do all parts) sound different.

Rob

"So back to my original premise, the only way to truly measure equipment is using scientific measurement. INDIVIDUALS CANNOT HEAR WHAT WE CANNOT MEASURE. Not with the precision instrumentation we have today."

mg

Doing work for and with test labs, they would disagree with you. This is why the testing world has been re-inventing themselves for many years. Testing isn't about the measurement, and building the product based on, testing is about inventing (with the help of equipment), using the human experience as part of the equation. The science programs I have been a part of, use the equipment to reverse engineer, but are never 100%. Many times not even close. I did testing for IBM some years ago. Not one of the tests read the same after vibratory adjustments were made to the test equipment itself.

My suggestion is this Rob

There's a world of difference when we look at this hobby as distortion and less distortion vs out-of-tune and in-tune. The point is obviously the "variables", and what is the logical science of variables? Tuning! Tune your guitar with the auto tune, now tune it by ear. What do they both have in common? The variable adjustability. This is where you find the answer to accuracy.

Also might I make this comment for good measure. If you have the basic audio testing equipment, used in an in-home environment, your measurements could easily be off by 20% accuracy. We have designed not only testing buildings but the furniture housing the test equipment itself, and even the very best of gear needs constant calibration, because of the varying condition changes.

So accuracy to an audio designer is fairly far away from a real lab environment.

michael green
MGA/RoomTune
http://tuneland.techno-zone.net/

Nt