You are here

Log in or register to post comments
Stephen Mejias
Stephen Mejias's picture
Offline
Last seen: 1 month 4 weeks ago
Joined: Nov 7 2010 - 3:35pm
A New Angle to an Old Discussion

This came in today:


Quote:

Dear International Colleagues,
I believe you all are aware of the long standing dispute between the subjectivists and objectivists in Audio. I believe nothing new has been learned in these discussions for the last 30 years or so and they basically come down to the inability of people who hear sonic differences to prove this in an ABX-Test which is then often considered to be prove of the non-existence of sonic differences.

We are putting a new spin onto this by resorting to a different field. People who work with graphics have often seen colour charts that show minimal differences that are only apparent in comparison to the surrounding colours. If one was to identify an individual colour without the surrounding ones for comparison that would be mostly guess work.

In other words: Even though there is a subjective difference to the observers senses and an objective difference as the colours really are mixed differently that does mean it will always show in an ABX-Test. Thereby the claim that not passing an ABX-Test means that there is no real difference is void.

We have integrated this concept into a small flash program you can access at www.sieveking-sound.de/abx which will do a little test with you were you are asked to first make sure that you do see the difference between two colours and then have to identify them later on. I yet have to score perfect and more often than not my results would let some people claim that this was proove that there is no difference in the colours. Alas

Stephen Scharf
Stephen Scharf's picture
Offline
Last seen: 11 months 1 day ago
Joined: Nov 19 2008 - 9:36pm
Re: A New Angle to an Old Discussion

I don't really agree with his theory. Color vision, and the way we see color is based on the Retinex Theory developed by Edwin Land. Color vision is a complex "color lookup table" computation that the brain does and is not a result of the wavelengths that we see. In this sense, I don't think it is comparable to how we hear, which is based on wavelength. OTOH, how we distinguish differerences in timbre have not really been fully understood.

Still, I don't think the Retinex Theory of Color Perception means that we can't use double-blind testing for listening comparisons.

RGibran
RGibran's picture
Offline
Last seen: 2 years 1 day ago
Joined: Oct 11 2005 - 5:50pm
Re: A New Angle to an Old Discussion


Quote:
I scored a 9 out of 20, which the computer tells me is worse than what I would have scored had I simply guessed.

Which may be correct as that is what I did, selecting "B" everytime, and scored 11 out of 20.

I think I'll try it again selecting "A" this time.

RG

judicata
judicata's picture
Offline
Last seen: Never ago
Joined: Jun 26 2008 - 11:55am
Re: A New Angle to an Old Discussion

I appreciate the thought, but I think it illustrates something different.

I think one mistake both sides make is that an A/B test is not considered, in any field that I'm aware of, absolute proof. It is evidence. In some cases, the evidence is stronger than others. But even in drug trials, there may be a drug that helps 2% of the population with condition Y, and the drug may not get approved or manufactured because the A/B results are inconclusive. Perhaps the drug even HURT another 4% of the same population.

In other words, when either (a) pro-ABXers say it is an absolute proof, or (b) anti-ABXers assume that ABXers believe it is an absolute proof, the discussion becomes really useless.

Of course, I've said this for as long as I've been in these forums. A/B tests are useful but not absolute. I promote A/B tests because I belive the more evidence we have of something the better; I also believe that not too much stock should be put into the A/B tests. For example, if an A/B test was inconclusive (i.e. no one could tell a difference), but YOU think it sounds better, it doesn't mean those people or deaf; likewise, it doesn't mean you are imagining things.

There are also thresholds. If I A/B'd black and white and color, I could definitely tell a difference.

Why do A/B tests at all? Like I said, it is information and information is good--it just shouldn't be misused.

Lick-T
Lick-T's picture
Offline
Last seen: Never ago
Joined: May 14 2006 - 8:04pm
Re: A New Angle to an Old Discussion


Quote:
Still, I don't think the Retinex Theory of Color Perception means that we can't use double-blind testing for listening comparisons.

No, it doesn't show we can't use ABX. It shows how hard it is to perceive actual difference within the context of ABX.
I got 7/20 right. I blame my laptop monitor and my complete and utter lack of training in seeing colors. I

smejias
smejias's picture
Offline
Last seen: 3 years 10 months ago
Joined: Aug 25 2005 - 10:29am
Re: A New Angle to an Old Discussion


Quote:
No, it doesn't show we can't use ABX. It shows how hard it is to perceive actual difference within the context of ABX.


Yes.


Quote:
I got 7/20 right. I blame my laptop monitor and my complete and utter lack of training in seeing colors. I
Jan Sieveking
Jan Sieveking's picture
Offline
Last seen: Never ago
Joined: Jul 24 2009 - 8:54am
Re: A New Angle to an Old Discussion

Hi this is Jan Sieveking who created that test.
I am quite surprised for what many people take this simple test. It is not making a statement as such but only is designed to show that even though there is an objective difference that is actually perceived by the test subjects (that is you doing the test!) that does not mean that difference will be perceived in this kind of ABX-Test.

It does not say that blind testing and A/B comparisons don't have good uses but you cannot rely on either one to get a conclusive picture.

As I am leaving for vacation I will not be around to answer many questions personally. Regardless of that please feel free to tell you friends about this little test.

And above all: Have fun listening to good sounding music.

Jan

ncdrawl
ncdrawl's picture
Offline
Last seen: 3 years 5 months ago
Joined: Oct 18 2008 - 9:18am
Re: A New Angle to an Old Discussion

I got 18 out of 20 and 16 of 20.

I gots a good eye.

RGibran
RGibran's picture
Offline
Last seen: 2 years 1 day ago
Joined: Oct 11 2005 - 5:50pm
Re: A New Angle to an Old Discussion

WOW! Must have been using the green color.

RG

ncdrawl
ncdrawl's picture
Offline
Last seen: 3 years 5 months ago
Joined: Oct 18 2008 - 9:18am
Re: A New Angle to an Old Discussion


Quote:
WOW! Must have been using the green color.

RG

I treated my monitor with the furutech demag and had Kait do the teleportation tweak to my DSL modem

Lick-T
Lick-T's picture
Offline
Last seen: Never ago
Joined: May 14 2006 - 8:04pm
Re: A New Angle to an Old Discussion


Quote:
I treated my monitor with the furutech demag and had Kait do the teleportation tweak to my DSL modem

D'oh! I shoulda done that! Good thinking.

smejias
smejias's picture
Offline
Last seen: 3 years 10 months ago
Joined: Aug 25 2005 - 10:29am
Re: A New Angle to an Old Discussion

Okay, I tried again, this time on my Mac, using the gray-blue squares and maximizing the color difference. I scored a 17/20.

dbowker
dbowker's picture
Offline
Last seen: 3 months 1 day ago
Joined: May 8 2007 - 6:37am
Re: A New Angle to an Old Discussion


Quote:

Quote:
Still, I don't think the Retinex Theory of Color Perception means that we can't use double-blind testing for listening comparisons.

No, it doesn't show we can't use ABX. It shows how hard it is to perceive actual difference within the context of ABX.
I got 7/20 right. I blame my laptop monitor and my complete and utter lack of training in seeing colors. I

KBK
KBK's picture
Offline
Last seen: Never ago
Joined: Sep 30 2007 - 12:30pm
Re: A New Angle to an Old Discussion

I deal with colors every day. My Business partner in Goo systems, Steve Ginsberg, Owns and runs 'Tri-Art' the world's highest grade Artist Acrylics coatings factory. Insane color. You have no idea how intense the pigmentation is.

The point is that colors have to tested beside one another and the eye is sharper than the instrument when it comes to getting it right. End of story. Over 20,000 different batches of paints and over 6,000 different sku's can be made at that factory. Damn -we know color. And the guy who makes the world's most pre-eminent industry calibration color charts, he and I BOTH agree that a notable important facet or two of the standards used in color theory are totally *screwt*. It is the knowledge of how it is screwed and what is specifically wrong...that we both closely hold, and we do not share.

Everyone else looks at color and calls themselves an expert. We build color - from the ground up. Not just any colors, but the world's finest. The complexities are, uhm, interesting. They'll teach you a lot of things you never thought you'd have to know or to become an expert in.

It is the male gene that possesses the color blindness, no female is color blind. IIRC, one out of every ...10 (?) posses some level of color blindness. The problem will be in the reds-orange-brown shades when blending. Thus these can be the problem areas we males will show in such tests.

High brightness can maximize the contrast entering the eye, this has an effect of increasing the apparent range of the eye's capacity for the recognition of subtlety. So if you have a poor monitor, view in it's most linear range (this needs to be determined with the controls initially set at 50% or so-then vary them) in a dark room.

Monty
Monty's picture
Offline
Last seen: Never ago
Joined: Sep 16 2005 - 6:55pm
Re: A New Angle to an Old Discussion

Which color sounds the best?

KBK
KBK's picture
Offline
Last seen: Never ago
Joined: Sep 30 2007 - 12:30pm
Re: A New Angle to an Old Discussion


Quote:
Which color sounds the best?

Depends on the mood, I'd hazard, and what direction you are trying to achieve.

mrlowry
mrlowry's picture
Offline
Last seen: 2 years 1 month ago
Joined: May 30 2006 - 1:37pm
Re: A New Angle to an Old Discussion


Quote:
Which color sounds the best?

Color affects mood which could change our perception of the sound. "Honey, you can't paint the living room purple. It'll make my system sound too royal." Now how's that for tweaky

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
It is the male gene that possesses the color blindness, no female is color blind. IIRC, one out of every ...10 (?) posses some level of color blindness.

Small quibble. The genes for color-blindness are recessive, and on the X chromosome. BUT they are not on the 'y' chromosome.

This means that an XY is MUCH more likely to be color blind, but people with two defective 'X's can be color-blind, it's just much less likely.

There are also some few people, who get two different copies of one of the pigments, and as a result wind up with a kind of 4-color vision, rather than 3. Now, this requires 2 X chromosomes, so guys are sorta ruled out here... (it is still possible, via some mechanisms that you don't want to have) It's very unusual, but has been confirmed in a few people. It's not clear if this is an advantageous thing, or 'just is'.

But, the association of colorblindiness with guys is because the 'y' chromosome does not have that gene at all, in any form, so any defect in the 'x' remaining will cause some issue.

krabapple
krabapple's picture
Offline
Last seen: Never ago
Joined: Sep 7 2005 - 8:10pm
Re: A New Angle to an Old Discussion

Since
1) no sensible person has ever claimed that DBTs -- or any science -- can *absolutely* 'prove a negative' (DBT results are explictly analyzed in terms of probability, and the probability can approach, but not reach, 1 or 0), and
2) no SP denies that training of subjects is a Good Thing for optimizing discrimination during DBT of audio, and
3)no SP denies we can usually measure some difference between two audio devices/sources, even when there is no DBT evidence of audible difference

what is the point of this attempt to analogize visual with audio perception?

If the thesis is that the extant DBTs are designed poorly in that the 'right'(sufficiently sensitive) subjects aren't being used -- even though self-proclaimed 'golden ears' and recording industry pros traditionally *have* taken part in DBTs of various As and Bs (and have, on more than one occasion, obtained results consistent with 'no audible difference' for them) -- then why doesn't the high-end culture design and conduct its own high-quality DBTs and lay the matters to rest with a slew of better-than-chance results for pairs of A/B heretofore considered 'audibly the same' by skeptics? Wouldn't this be more definitive than tortuous Harley-esque arguments and 'new angles' about why DBT just doesn't/shouldn't/can't work?

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
then why doesn't the high-end culture design and conduct its own high-quality DBTs and lay the matters to rest with a slew of better-than-chance results for pairs of A/B heretofore considered 'audibly the same' by skeptics? Wouldn't this be more definitive than tortuous Harley-esque arguments and 'new angles' about why DBT just doesn't/shouldn't/can't work?

My suspicioun is that they've tried them and found themselves wanting.

Scott Wheeler
Scott Wheeler's picture
Offline
Last seen: Never ago
Joined: Sep 3 2005 - 7:47pm
Re: A New Angle to an Old Discussion

I suspect that such a claim deliberately paints with too broad a brush just to fish for a fight. What? Revel speakers aren't high end? too bad about that. I was thinking about auditioning one of their subwoofers. What about Dunlavy speakers? They were high end in my book. And who here really knows what's going down in the R&D departments at the various high end manufacturers? Some times it seems that some folks are so desperate to hold on to old tired arguments that they have to invent enemies to battle with. don't we have enough such folks at the lunitic fringes?

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
I suspect that such a claim deliberately paints with too broad a brush just to fish for a fight. What? Revel speakers aren't high end? too bad about that. I was thinking about auditioning one of their subwoofers. What about Dunlavy speakers? They were high end in my book. And who here really knows what's going down in the R&D departments at the various high end manufacturers? Some times it seems that some folks are so desperate to hold on to old tired arguments that they have to invent enemies to battle with. don't we have enough such folks at the lunitic fringes?

Neither Kevin (Voecks) or John (Dunlavy) are high-end in philosophy, both are hard-nosed scientists. I'm quite comfortable saying that, having conversed on the subject with both of them.

I'm not sure what you're saying in terms of "a broad brush", perhaps we regard different things as 'the high end'.

geoffkait
geoffkait's picture
Offline
Last seen: 2 hours 34 min ago
Joined: Apr 29 2008 - 5:10am
Re: A New Angle to an Old Discussion

"Neither Kevin (Voecks) or John (Dunlavy) are high-end in philosophy, both are hard-nosed scientists."

By "hard-nosed scientists" I guess you mean those that think everything they don't believe in must be explained by the placebo effect?

from Dunlavy's anti-cable rant circa 1996:

"Are there really any significant audible differences between most cables that can be consistently identified by experienced listeners? The answer is simple: very seldom! Those who claim otherwise do not fully grasp the power of the old Placebo-Effect - which is very alive and well among even the most well-intentioned listeners. The placebo-effect renders audible signatures easy to detect and describe - if the listener knows which cable is being heard. But, take away this knowledge during blind or double-blind listening comparisons and the differences either disappear completely or hover close to the level of random guessing. Speaking as a competent professional engineer, designer and manufacturer, nothing would please me and my company's staff more than being able to design a cable which consistently yielded a positive score during blind listening comparisons against other cables. But it only rarely happens - if we wish to be honest!"

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
"Neither Kevin (Voecks) or John (Dunlavy) are high-end in philosophy, both are hard-nosed scientists."

By "hard-nosed scientists" I guess you mean those that think everything they don't believe in must be explained by the placebo effect?

Perhaps you could avoid lying about what I've said, Mr. Krait.

Scott Wheeler
Scott Wheeler's picture
Offline
Last seen: Never ago
Joined: Sep 3 2005 - 7:47pm
Re: A New Angle to an Old Discussion


Quote:

Quote:
I suspect that such a claim deliberately paints with too broad a brush just to fish for a fight. What? Revel speakers aren't high end? too bad about that. I was thinking about auditioning one of their subwoofers. What about Dunlavy speakers? They were high end in my book. And who here really knows what's going down in the R&D departments at the various high end manufacturers? Some times it seems that some folks are so desperate to hold on to old tired arguments that they have to invent enemies to battle with. don't we have enough such folks at the lunitic fringes?

Neither Kevin (Voecks) or John (Dunlavy) are high-end in philosophy, both are hard-nosed scientists. I'm quite comfortable saying that, having conversed on the subject with both of them.

I'm not sure what you're saying in terms of "a broad brush", perhaps we regard different things as 'the high end'.

To me high end is anything that tries to excel in sonic performance. So I would consider both of those designers and their products high end. The Dunlavy IVs Vs and VIs were pretty excellent speakers IMO. Definitely high end. I believe the IVs were the product of the year in Stereophile way back when.

smejias
smejias's picture
Offline
Last seen: 3 years 10 months ago
Joined: Aug 25 2005 - 10:29am
Re: A New Angle to an Old Discussion


Quote:
To me high end is anything that tries to excel in sonic performance.


I agree. Whenever this strange question comes up, I refer to JA's essay, The High End, Mid-Fi, & Pretend High End.

KBK
KBK's picture
Offline
Last seen: Never ago
Joined: Sep 30 2007 - 12:30pm
Re: A New Angle to an Old Discussion

Any place that moves in the dark outer edges where textbooks fear to tread, is going to inherently be seen by mainstream thinking as 'lunatic fringe'. This is the way it has always been. This is the way it will always be.

There are those who try to explore and some do it in different ways. One given orientation and execution is not necessarily any worse than any other method or philosophy.

The smart man understands when others think he is a lunatic, that he is very likely doing something inherently - right.

For the explorers of the edge will always be condemned by those who lack that capacity. This is part of the package that the intrepid explorer buys into. This is the way of the world.

I mention this as the world of High End Audio (via it's very nature) inherently moves into the cutting edge of multiple fields.

Scott Wheeler
Scott Wheeler's picture
Offline
Last seen: Never ago
Joined: Sep 3 2005 - 7:47pm
Re: A New Angle to an Old Discussion


Quote:
Any place that moves in the dark outer edges where textbooks fear to tread, is going to inherently be seen by mainstream thinking as 'lunatic fringe'. This is the way it has always been. This is the way it will always be.

There are those who try to explore and some do it in different ways. One given orientation and execution is not necessarily any worse than any other method or philosophy.

The smart man understands when others think he is a lunatic, that he is very likely doing something inherently - right.

For the explorers of the edge will always be condemned by those who lack that capacity. This is part of the package that the intrepid explorer buys into. This is the way of the world.

Luckily it just isn't such a serious thing with audiophiles. It's just a hobby. I would say that Charles Manson and Jim Jones were um...lunitic fringe dwellers. And they both thought they were doing something inherently right didn't they? Speaking of interpid explorers, I wonder how the folks on Hale Bob are doing. You think they still have their Nikes? the point being some who are viewed as lunitics actually are lunitics.

KBK
KBK's picture
Offline
Last seen: Never ago
Joined: Sep 30 2007 - 12:30pm
Re: A New Angle to an Old Discussion

OK.

I'll push back a bit.

Identifying high end audio designers with people who are killers and homicidal maniacs, simply illustrates your personal lack of capacity for discernment or correct thought that is required for balanced thinking.

To venture into similar labeling as are you, Scott, I'd venture that there is a strong possibility that you secure your place in the world (internal mental stability) via linear thinking & negative labeling as form of externalized control. ie, negative labeled projection as a form of world stabilization and control.

This is not 'good human' behavior, rather -- it is the opposite. To me, this is very sick. Often, (OK, all I've seen so far) this appears to be the center of your arguments. I am sorry Scott, but after many viewings of your posts in argumentative subjects, this is always what seems to be coming up on the forum, with regard to your corner of expression.

Scott Wheeler
Scott Wheeler's picture
Offline
Last seen: Never ago
Joined: Sep 3 2005 - 7:47pm
Re: A New Angle to an Old Discussion


Quote:
OK.

I'll push back a bit.

Identifying high end audio designers with people who are killers and homicidal maniacs, simply illustrates your personal lack of capacity for discernment or correct thought that is required for balanced thinking.

Mine or yours? Where did I compare Manson to Bill Johnson? Where did I compare Jim Jones to Richard Vandersteen?


Quote:
To venture into similar labeling as are you, Scott, I'd venture that there is a strong possibility that you secure your place in the world (internal mental stability) via linear thinking & negative labeling as form of externalized control. ie, negative labeled projection as a form of world stabilization and control.

Wow you got all that from me pointing out that some folks who are labeled lunitics are done so for good reasons?


Quote:
This is not 'good human' behavior, rather -- it is the opposite. To me, this is very sick. Often, (OK, all I've seen so far) this appears to be the center of your arguments. I am sorry Scott, but after many viewings of your posts in argumentative subjects, this is always what seems to be coming up on the forum, with regard to your corner of expression.

Really? *I'm* sick for labeling Jim Jones and Charles Manson as lunitics? Perhaps I have misjudged them?!?

satkinsn
satkinsn's picture
Offline
Last seen: 2 years 5 months ago
Joined: Aug 19 2008 - 4:23am
Re: A New Angle to an Old Discussion

Interesting.

I've posted here before that I am a very reluctant audiophile, that I listened to music on a $99 boombox for years, and that what happened to me was mp3 files/aac files - even good ones - started to annoy me. It was like something was reaching into my head and irritating it, if that's not too far fetched.

So I ended up buying some decent, though low end equipment to try to vanquish the problem and get back to listening.

My point being: I have nothing to do with visual art and, in fact, spend my day surrounded by tv screens and monitors. They almost all pump out lousy cable analog signals. Plus, my eyes aren't that good.

Took the test, using the cyan color, with the difference set at 5. Got 15 out of 20 without thinking about it. And here's the thing - my head felt exactly like I feel when I'm listening to compressed music, a kind of buzzy irritation.

I also suffer from migraines that got more severe about the time I started having my mp3/aac problem, and this test, like compressed music sometimes, seems to want to trigger one.

I have no idea whether I'm just deluding myself, or whether I'm mistaking coincidence for cause, or whatever. But it's strange, and it feels real.

s.

FWIW, I quickly ran the test four more times, scoring 10/20, 12/20, 6/20 and 14/20.

The ones where I got scores that were no better (or much worse) than random, I really couldn't see much difference. The last time, (a very light yellow) the difference was obvious. In other words, I knew when I was guessing, and I knew when I wasn't.

Of course, it's also possible I'm training myself to see small differences.

absolutepitch
absolutepitch's picture
Offline
Last seen: 10 months 2 weeks ago
Joined: Jul 9 2006 - 8:58pm
Re: A New Angle to an Old Discussion


Quote:

Quote:
--------------------------------------------------------------------------------

then why doesn't the high-end culture design and conduct its own high-quality DBTs and lay the matters to rest with a slew of better-than-chance results for pairs of A/B heretofore considered 'audibly the same' by skeptics? Wouldn't this be more definitive than tortuous Harley-esque arguments and 'new angles' about why DBT just doesn't/shouldn't/can't work?

--------------------------------------------------------------------------------

My suspicioun is that they've tried them and found themselves wanting.

Could be in some cases. Not so in others.

DBT (or SBT) have their place, and used correctly is a powerful tool. A positive result means the test detected a difference, a result differs from that by chance alone. A negative result means the test did not detect a difference from chance alone. As 'krabapple' says, it cannot prove a negative (and I think nothing will).

I did have a friend correctly identify 100% of the time the difference between the two states A and B (different situation from above) in a SBT situation, for a tweak that some 'objectivists' have dismissed in the past as inaudible, whereas 'subjectivists' have claimed that it makes a difference.

This latter experience suprised my friend, who could not believe what he was hearing, and reliably described it each time I switched A to B, or back to A and so forth (I even tried not switching once and he got it right). Later I told him what the tweak was. I was performing this test to see if he could hear the same type of changes I had heard by myself in a sighted comparison before ("A") and after ("B") the tweak.

So my take on this is that things are not always what we think it is, regardless of all the theory/practice I learned from school and in the field. In the above latter case, the sighted test was confirmed by the SBT. What is remarkable is that I had made some 'other' tweaks (prior to the A/B test), that are still regarded by many 'objectivists' as nonsense. Yet before those 'other' tweaks, I could not hear any difference between A and B states. But after those tweaks I could readily identify A vs. B states (suprise!). So, the SBT test indirectly confirms the effectiveness of the earlier tweaks that enabled the distinguishing of A vs. B.

As any scientist/engineer would state this was not a rigorous controlled test, although I remain convinced of its results, just as convinced as any musician might identify the differences between two violins for example the same way I identified the before and after 'other' tweaks as well as the A vs. B.

Scott Wheeler
Scott Wheeler's picture
Offline
Last seen: Never ago
Joined: Sep 3 2005 - 7:47pm
Re: A New Angle to an Old Discussion


Quote:

Quote:

Quote:
--------------------------------------------------------------------------------

then why doesn't the high-end culture design and conduct its own high-quality DBTs and lay the matters to rest with a slew of better-than-chance results for pairs of A/B heretofore considered 'audibly the same' by skeptics? Wouldn't this be more definitive than tortuous Harley-esque arguments and 'new angles' about why DBT just doesn't/shouldn't/can't work?

--------------------------------------------------------------------------------

My suspicioun is that they've tried them and found themselves wanting.

Could be in some cases. Not so in others.

DBT (or SBT) have their place, and used correctly is a powerful tool. A positive result means the test detected a difference, a result differs from that by chance alone. A negative result means the test did not detect a difference from chance alone. As 'krabapple' says, it cannot prove a negative (and I think nothing will).

I did have a friend correctly identify 100% of the time the difference between the two states A and B (different situation from above) in a SBT situation, for a tweak that some 'objectivists' have dismissed in the past as inaudible, whereas 'subjectivists' have claimed that it makes a difference.

This latter experience suprised my friend, who could not believe what he was hearing, and reliably described it each time I switched A to B, or back to A and so forth (I even tried not switching once and he got it right). Later I told him what the tweak was. I was performing this test to see if he could hear the same type of changes I had heard by myself in a sighted comparison before ("A") and after ("B") the tweak.

So my take on this is that things are not always what we think it is, regardless of all the theory/practice I learned from school and in the field. In the above latter case, the sighted test was confirmed by the SBT. What is remarkable is that I had made some 'other' tweaks (prior to the A/B test), that are still regarded by many 'objectivists' as nonsense. Yet before those 'other' tweaks, I could not hear any difference between A and B states. But after those tweaks I could readily identify A vs. B states (suprise!). So, the SBT test indirectly confirms the effectiveness of the earlier tweaks that enabled the distinguishing of A vs. B.

As any scientist/engineer would state this was not a rigorous controlled test, although I remain convinced of its results, just as convinced as any musician might identify the differences between two violins for example the same way I identified the before and after 'other' tweaks as well as the A vs. B.

IMO Blind tests single or double done by hobbyists are anecdotal either way. Don't think I am dismissing your tests. If you found them informative that is great but as fara s "proof" of anything I dismiss all such tests. they *might* be better than sighted tests. They might not! Hard to tell inthe hands of hobbyists. Personally I do find them very useful in my audition process but I would never confuse my audition proces with scientific research. Nor do i think one has to do scientific research to justify their preferences. Quite the opposite. If someone thinks something sounds better or worse. They are right. At least for their purposes a a hobbyist.

ncdrawl
ncdrawl's picture
Offline
Last seen: 3 years 5 months ago
Joined: Oct 18 2008 - 9:18am
Re: A New Angle to an Old Discussion


Quote:
for a tweak that some 'objectivists' have dismissed in the past as inaudible

what tweak would that be?

Drtrey3
Drtrey3's picture
Offline
Last seen: 4 months 1 week ago
Joined: Aug 17 2008 - 2:52pm
Re: A New Angle to an Old Discussion

I got 16 out of 20. but then I calibrated the test images until I could just distinguish them when side by side. That would certainly have a HUGE impact on the ability or inability to distinguish them in the testing phase.

Trey

absolutepitch
absolutepitch's picture
Offline
Last seen: 10 months 2 weeks ago
Joined: Jul 9 2006 - 8:58pm
Re: A New Angle to an Old Discussion


Quote:

Quote:
--------------------------------------------------------------------------------

for a tweak that some 'objectivists' have dismissed in the past as inaudible

--------------------------------------------------------------------------------

what tweak would that be?

It was an experiment whether speaker fuses and the type used affected the sound. Sure enough each type sounds different. That was a comparison between a fuse and a wire in place of the fuse. This comparison was posted in Tweaks and Tips some time ago; I've forgotten whether it was before I joined the forum (anonymous) or not.

geoffkait
geoffkait's picture
Offline
Last seen: 2 hours 34 min ago
Joined: Apr 29 2008 - 5:10am
Re: A New Angle to an Old Discussion

"It was an experiment whether speaker fuses and the type used affected the sound. Sure enough each type sounds different. That was a comparison between a fuse and a wire in place of the fuse."

Wonder if anyone ever compared fuses for picture/color quality? DVD players may not have fuses, not sure, but high end plasma and projection systems most likely do....

absolutepitch
absolutepitch's picture
Offline
Last seen: 10 months 2 weeks ago
Joined: Jul 9 2006 - 8:58pm
Re: A New Angle to an Old Discussion

I've not tweaked TVs yet, but I know someone who did. He says the picture is amazing afterward. I don't know what he did.

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
Any place that moves in the dark outer edges where textbooks fear to tread, is going to inherently be seen by mainstream thinking as 'lunatic fringe'. This is the way it has always been. This is the way it will always be.


It certainly describes how standard signal processing initially dealt with the idea of perceptual coding.

Quote:

The smart man understands when others think he is a lunatic, that he is very likely doing something inherently - right.


Sorry, no, the smart man knows that most lunatics are in fact simply lunatics.

The smart man knows that testability, repeatability, confirmation, attempts at falsification, etc, are the key of figuring out which are lunatics and which are the 'lunatic you're waiting for'.

Quote:

I mention this as the world of High End Audio (via it's very nature) inherently moves into the cutting edge of multiple fields.

The problem with audio in general (and one that relates directly to the initial issues with perceptual coding, is that many people either do not do well when reaching out of their speciality (I think we've seen an example of that lately), or avoid looking to other disciplines.

Audio by its nature involves perception, psychology, psychoacoustics, biology, physical acoustics, signal processing (both analog and digital), physics of materials, and a bunch of mathematics. A lot of people get lost in the middle of all that. It is by definition a multidisciplinary field, and one that can not easily be split off into its own field, because it requires detailed understanding in a number of fairly complex fields.

Into this we have the facts regarding human memory and auditory processing, and we have the makings of a problem.

Buddha
Buddha's picture
Offline
Last seen: 2 years 11 months ago
Joined: Sep 8 2005 - 10:24am
Re: A New Angle to an Old Discussion

Man, was that interesting!

In the light green, I was random. In the brownish, 18/20.

This raises many questions.

1) Do others score differently with different hues?

2) Is there a hue that is 'easiest' to discern differences?

3) What is the degree of impact of varying the time intervals?

4) Have they done ABX where the colors are side by side and all the test taker has to do is say, "Same/Different?"

4a) If so, we should invent a new term for ABX testing. Instead of 'instanatenous,' we should call it 'simultaneous' ABX testing. Of course, the audio equivalent of this would be very difficult.

6) Sense memory, of course, comes into play. Did anyone else notice 'test drift' while taking the test, having to ask himself which color was which, even if a difference could be detected? Sometimes, it felt like I 'forgot' which color was "A" or "B."

7) Do test results vary by age or gender of the test taker?

8) Most of the tests seem to be of the shorter wavelength colors, only once in the red category for me. Is the test biased toward blue/green, or yellow, and shorter?

____________

I also wonder how this test would go if we had to assign vocabulary and name these colors.

I ask because maybe I can tell certain ones apart, but in real life I often 'misname' colors.

I recently rented a home that's kind of olive drab, but told my wife and others in giving directions to the house to look for the pink one.

When someone sat me down and pointed out the actual color of the house, I could 'get it,' but left to my own for 'naming' the color, it was different.

Hard to describe what I really mean. I could see it, but in my brain, the words were stored in overlapping areas.

Maybe it's tone/tint/shade related more that color for some?

Same thing has happened in my past love life!

SAS Audio
SAS Audio's picture
Offline
Last seen: 7 months 1 week ago
Joined: Jun 6 2007 - 6:56am
Re: A New Angle to an Old Discussion


Quote:
Man, was that interesting!

In the light green, I was random. In the brownish, 18/20.

This raises many questions.

1) Do others score differently with different hues?

2) Is there a hue that is 'easiest' to discern differences?

3) What is the degree of impact of varying the time intervals?

4) Have they done ABX where the colors are side by side and all the test taker has to do is say, "Same/Different?"

4a) If so, we should invent a new term for ABX testing. Instead of 'instanatenous,' we should call it 'simultaneous' ABX testing. Of course, the audio equivalent of this would be very difficult.

6) Sense memory, of course, comes into play. Did anyone else notice 'test drift' while taking the test, having to ask himself which color was which, even if a difference could be detected? Sometimes, it felt like I 'forgot' which color was "A" or "B."

7) Do test results vary by age or gender of the test taker?

8) Most of the tests seem to be of the shorter wavelength colors, only once in the red category for me. Is the test biased toward blue/green, or yellow, and shorter?

____________

I also wonder how this test would go if we had to assign vocabulary and name these colors.

I ask because maybe I can tell certain ones apart, but in real life I often 'misname' colors.

I recently rented a home that's kind of olive drab, but told my wife and others in giving directions to the house to look for the pink one.

When someone sat me down and pointed out the actual color of the house, I could 'get it,' but left to my own for 'naming' the color, it was different.

Hard to describe what I really mean. I could see it, but in my brain, the words were stored in overlapping areas.

Maybe it's tone/tint/shade related more that color for some?

Same thing has happened in my past love life!

Interesting and very good questions Buddha. I think there are many similarities between sight perception and sound perception. Also Time frame and repeatability are two interesting thoughts since basic variabilities or flaws exist that can corrupt the input data every time a test is performed. Even though the math may show great confidence, the results of each test would be worthless because of corrupted data. So one can repeatably test and come out with repeatably worthless conclusions every time because of one or more basic flaws in perception not accounted for, and yet think his/her conclusions are correct because the conclusions consistently "lined up".

With that said, it is possible to draw some conclusions as in the medical fields although no one states this is "fact" but rather, like Dr. Timothy Johnson often states, it is believed, the study suggests etc.

Jan Sieveking
Jan Sieveking's picture
Offline
Last seen: Never ago
Joined: Jul 24 2009 - 8:54am
Re: A New Angle to an Old Discussion

Hi Everybody,
it is time to give some input from the person who designed the test.

First of all I am very pleased by the discussion culture I have observed here. It seems it is indeed possible to discuss something without getting hateful about it and secondly a few very good questions have been asked. I have to admit a German forum with pretty much exactly the same starting information quickly degraded into the topic of "what is this guys plan and what can we do to prevent it". People are funny animals.

So back to the topic. The one and only thing this version of the ABX-Test does prove is that it is not easy to distinguish two similar but not identical stimuli over time when they are not presented simultaneously. It does not say that some kind of blind test might not be worthwhile but these should probably be carried out in some other way than this test. Unfortunately this version is very similar to how some people think a test should be designed. Especially in internet forums.

We did actually find that if you had no interval between the different colours one would easily see the "change" on the screen and this even works well with very low colour differences. However, we do not know if this should be true for minimal changes in recorded music and there is little merit in knowing if a change has happened in itself when the actual question would be if you feel a change in your musical enjoyment.

Jan Sieveking

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
First of all I am very pleased by the discussion culture I have observed here. It seems it is indeed possible to discuss something without getting hateful about it and secondly a few very good questions have been asked. I have to admit a German forum with pretty much exactly the same starting information quickly degraded into the topic of "what is this guys plan and what can we do to prevent it". People are funny animals.

So back to the topic. The one and only thing this version of the ABX-Test does prove is that it is not easy to distinguish two similar but not identical stimuli over time when they are not presented simultaneously. It does not say that some kind of blind test might not be worthwhile but these should probably be carried out in some other way than this test. Unfortunately this version is very similar to how some people think a test should be designed. Especially in internet forums.

We did actually find that if you had no interval between the different colours one would easily see the "change" on the screen and this even works well with very low colour differences. However, we do not know if this should be true for minimal changes in recorded music and there is little merit in knowing if a change has happened in itself when the actual question would be if you feel a change in your musical enjoyment.

Jan Sieveking

You do realize that your test has absolutely no connection to auditory perception, yes?

Just have to point that out. Auditory perception by its very nature has a time-domain aspect, as the ear is a frequency analyzer. There is no such analyzer in the visual pathways. What's more, while pretty much everyone with "normal" hearing has about the same kind of frequency analysis in their ears, color vision is rather more variable.

In sound, things have to be presented adjacent in time, it's the only way to get around the shortness of primary loudness memory. The various level-roving experiments (Hall, Allen, others) show this plain as day.

Delays between listening to A and B will simply not provide ultimate sensitivity.

And, of course, using DBT's (not ABX tests, but rather signal-detection tests, as appropriate to the stimulii, although ABX tests will work given enough subject training, and yes, that has been shown) it is clear, for instance, that DBT's will detect sounds right down into the actual limits of physics.

So, there is verification that ABX and DBT's in general work very, very well on auditory stimulii. Tests in vision do not apply, except in that they do show the problems that arise once you get away from basic peripheral sensitivity and into the CNS, where all the rules change when the subject decides they need to change.

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
4a) If so, we should invent a new term for ABX testing. Instead of 'instanatenous,' we should call it 'simultaneous' ABX testing. Of course, the audio equivalent of this would be very difficult.


Here you point out a basic difference in auditory vs. visual stimulii. It is possible to have a stationary visual stimulus. A stationary audio stimulus is called "air pressure".

There is about a 200 millisecond (at best) window in which one can compare partial loudnesses of two signals. This is why clickless instantaneous switching is required for any sensitive ABX test. It just goes with the territory.

Quote:

8) Most of the tests seem to be of the shorter wavelength colors, only once in the red category for me. Is the test biased toward blue/green, or yellow, and shorter?


Well, red/green distinctions are about half as sensitive as Blue/green or blue/red distinctions, due to the way the red and green pigments overlap spectrally in the eye.

Drtrey3
Drtrey3's picture
Offline
Last seen: 4 months 1 week ago
Joined: Aug 17 2008 - 2:52pm
Re: A New Angle to an Old Discussion

"You do realize that your test has absolutely no connection to auditory perception, yes?"

Good point. I use my brain to process visual sensory input and some people evidently use a horse's ass to listen.

Trey

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:

"You do realize that your test has absolutely no connection to auditory perception, yes?"

Good point. I use my brain to process visual sensory input and some people evidently use a horse's ass to listen.

Trey

I take it that this is your response to an informed reply.

Now, please explain how you can argue that the same peripheral level effects one observes in vision also operate in audition.

Got any evidence?

Note, PERIPHERAL. I'm presuming you know what that means.

Buddha
Buddha's picture
Offline
Last seen: 2 years 11 months ago
Joined: Sep 8 2005 - 10:24am
Re: A New Angle to an Old Discussion


Quote:

Quote:

"You do realize that your test has absolutely no connection to auditory perception, yes?"

Good point. I use my brain to process visual sensory input and some people evidently use a horse's ass to listen.

Trey

I take it that this is your response to an informed reply.

Now, please explain how you can argue that the same peripheral level effects one observes in vision also operate in audition.

Got any evidence?

Note, PERIPHERAL. I'm presuming you know what that means.

There is alot of commonality.

Both audition and visual memory show decay in accuracy as the interval between stimuli is increased.

Both types of tests likely have a significant component of test fatigue, or drift over multiple trials. Be it tones or hues, the test taker can drift off point without reinforcement of the original test stimuli.

Both may be 'frequency' dependent - demonstrating different sensitivities at different sonic or optic 'wavelengths.'

Both can be affected by the medium thorugh which the stimuli are presented - your computer monitor or mine; your Hi Fi or mine - and other ambient 'noises' may alter test results.

While not 100% correlated, the visual tests point out many things in common with the travails of testing auditory phenomena.

Similar outcomes could probably be demonstrated in wine taste tests, as well - and 'smell' fatigue is well described in subjective testing. Lots in common all around, and great to compare as a way of demonstrating the strengths and weaknesses of testing styles for different sense events.

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
There is alot of commonality.


After you leave the periphery, there is some.

Quote:

Both audition and visual memory show decay in accuracy as the interval between stimuli is increased.


At entirely different rates via what pretty much appears to be from different mechanisms, for the first level of analysis (retinal vs. cochlear).


Quote:

Both types of tests likely have a significant component of test fatigue, or drift over multiple trials. Be it tones or hues, the test taker can drift off point without reinforcement of the original test stimuli.


Which is why tests are partitioned to short parts, and why tests have controls.

Quote:

Both may be 'frequency' dependent - demonstrating different sensitivities at different sonic or optic 'wavelengths.'


Irrelevant.

You can detect flicker at most at 75Hz or so.

The detection of color is not "frequency analysis" in any sense like the cochlea works for sound. The cochlea IS a filterbank, it is a mechanical filterbank (yes, implimented in biology) that separates frequencies mechanically.

The eye does nothing of the sort. It has 4 receptors, 'y' and RGB. These do not have a high frequency response, nor do they have any substantial frequency selectivity, rather the rhopsodins in the eye have wide spectral sensitivity. All in all, the eye can detect 3 (or 4 in some few people)(not counting rods, or 'y' which is common to most any sighted person) different sets of wavelengths, ranging over an octave. From this, via secondary methods, the eye can resolve color as we know it.

The ear resolves frequency over at least 10 octaves, and does so directly, not by differential excitation. From that, it has the pitch and loudness sensitivity we know.

The eye has, at any one time (i.e. not counting adaptation), a spatial dynamic range of about a factor of 1000 in energy. This changes over time by a lot to get to a factor of 1,000,000,000 or so over a period of at least 30 minutes. This over an input frequency range of an octave in terms of light wavelengths.

The ear adapts in 50 milliseconds over that same factor of 1,000,000,000. At any given time, it has a spectral dynamic range of at least 60dB at the instant. (which spreads over the 120dB or so range we know about over short periods of time, on the order of seconds) This over 10 octaves of time-domain information.

The eye is simply not a frequency analyzer. It does not work that way, it uses a couple of differential sensors that create color as we know it from a couple of happy accidents of pigment selection.

The ear is intrinsically a frequency analyzer.

If you use amplitude distortion in an image, the image remains the same, you might change the contrast or brightness a bit, but it's not going to change an 'a' to a 'b'. Huge amplitude distortions are in fact deliberately applied during photo production, etc, to good end.

If you do amplitude distortions for a sound, you create new sounds. You create new frequencies. You HEAR new frequencies. If I add x*x to the audio signal, you'll hear both the second harmonic and all the intermod products. If I add x*x to the image amplitude you won't notice it until it's truly enormous amounts, and then you'll still know what the picture is, and it won't look ridiculous. At the same level the sound would be unbearable and unrecognizable.

Why? Because the eye does not analyze frequency, either spatially or in the time domain INTRINSICALLY, and the ear is intrinsically a time-domain frequency analyzer. The two peripheral items are extremely different. The second harmonic in an image does not change the 'Information', but the second harmonic in a sine wave is a new sound at a new pitch.

As I said before, "stationary sound" is called barometric pressure. On the other hand, you CAN have a still-frame image and comprehend it.

Quote:

Both can be affected by the medium thorugh which the stimuli are presented - your computer monitor or mine; your Hi Fi or mine - and other ambient 'noises' may alter test results.


Irrelevant and true of everything on the planet.

Quote:

While not 100% correlated, the visual tests point out many things in common with the travails of testing auditory phenomena.


Not at all. If I make a flicker at 300Hz, you can't see it.
If I try to get a 60dB dynamic range from the eye at one point, even a cataract-free youngster won't be able to do it.

Quote:

Similar outcomes could probably be demonstrated in wine taste tests, as well - and 'smell' fatigue is well described in subjective testing. Lots in common all around, and great to compare as a way of demonstrating the strengths and weaknesses of testing styles for different sense events.

Actually, not at all.

Smell and taste receptor fatigue is completely different and even slower than the eye. Some smell and taste receptors saturate and then will not respond for minutes or hours. What's more, there are 300+ different smell receptors known, and any one person appears to get somewhere around 100 of them, some people substantially more, some substantially less.

This is entirely unlike both vision and sound, wherein there are at least commonly agreed perceptions. If I give you a wine made of a "foxy" grape, 17% or so of the population will find it repugnant, and the rest won't ever notice, for instance. This is true of many odors. Some other examples are unscented deodorants and some chemicals that propriety suggests I leave unmentioned.

The ONLY commonality is when you get past the periphery. The problem is that the periphery is so different that the rules change entirely.

The commonality is that once you've gotten into the CNS, you can direct your attention toward or away from any given kind of sensation.

This, easily, can and does create two different memories and resulting sensations when sensing the exact same stimulus twice.

And that, indeed, is where many people get into a state of chaos in regard to auditory perception.

www.aes.org/sections/pnw/ppt.htm has a deck on the differences between sight and vision. You might want to check it out.

Buddha
Buddha's picture
Offline
Last seen: 2 years 11 months ago
Joined: Sep 8 2005 - 10:24am
Re: A New Angle to an Old Discussion

JJ, are you trying to channel Cliff Clavin or are you just being a stone bitch?

Stop with the pedantry, already!

Jesus Christ. (And I don't mean that in a good way.)

Go learn about vision - 'happy accident.'

Go to bed.

j_j
j_j's picture
Offline
Last seen: 2 years 4 months ago
Joined: Mar 13 2009 - 4:22pm
Re: A New Angle to an Old Discussion


Quote:
Go learn about vision - 'happy accident.'

Yeah, go check out how mammalian color vision has changed over time.

Yep, our color vision is better than most apes, etc. Why? Classic transcription error on a repeated chromosome.

Geeze, dude, what's got your socks in a knot, anyhow? The idea that a color test like the one above says much about audition is simply a wrong idea. 'taint even close to that simple.

Buddha
Buddha's picture
Offline
Last seen: 2 years 11 months ago
Joined: Sep 8 2005 - 10:24am
Re: A New Angle to an Old Discussion


Quote:

Quote:
Go learn about vision - 'happy accident.'

Yeah, go check out how mammalian color vision has changed over time.

Yep, our color vision is better than most apes, etc. Why? Classic transcription error on a repeated chromosome.

Geeze, dude, what's got your socks in a knot, anyhow? The idea that a color test like the one above says much about audition is simply a wrong idea. 'taint even close to that simple.

The point is, calling vision a happy accident is idiotic or subidiotic in terms of differentiating it from any attribute.

What, then, is audition, intelligent design?

There are grat degrees of analogy in sensory perception and processing.

"Audition is blah blah blah" and visual perception is a 'happy accident' is dumb beyond me being able to excuse as too much Barolo for JJ.

One is a "classic transcription error" and the other is?

Fuggin' Hell, now you need to start back 150 years ago with Sir Charles and catch up with how it all works.

Unless, of course, audition is a miracle of creation.

Pages

  • X
    Enter your Stereophile.com username.
    Enter the password that accompanies your username.
    Loading