You are here

Log in or register to post comments
Brucest
Brucest's picture
Offline
Last seen: 3 years 2 months ago
Joined: Feb 11 2006 - 7:31am
Re: Why is Blind Testing Such a Hot Button?

You seem to be conflating the idea of what a magazine should do to review equipment and what "science" or "engineering" broadly speaking "should do" to establish design criteria for equipment. Clearly it is in my view beyond the scope to a magazine to establish definitive design criteria. However, they could usefully establish that design criteria other than accuracy are important, or that certain types of accuracy are more important than others.

IMO this is critically important because years ago accuracy was deemed THE design criteria per se. Engineering and technology responded with equipment with increasing numbers of zeros to the right of the decimal point for very low prices in today's marketplace.

Thus one might conclude that if the highest of high enders can be specific (in terms of measured performance) about what they want, that engineering and technology can similarily provide that at very low prices. Yet no one seems to be able to do this. I don't understand why.

Years ago Bob Carver challenged the high end to the effect that he could make his el cheapo (relatively speaking) power amp sound like any mega buck of choice with similar design limits. As I recollect, Sterophile took him up on it, and Bob seemed to succeed aping a 15 or 20 times more expensive CJ. I never understood why the whole discussion faded away. The only "refutation" of Bob's achievement I read was that he couldn't do that with production equipment at his price point. That may well be so, but that's hardly a dismissal, since achieving the same at double or triple his price point was still a bargain. (Another rather silly point was made to the effect that Bob was "stealing" someone else's sound; his response was to design a super tube amp that remains the reference for some to this day, I think.)

I'm agnostic about whether the highest high enders are right, but I find it curious that many of their claims are unfazed by vast improvement in technology, and find their unwillingness or inability to state specificly what they want to be very puzzling.

nunhgrader
nunhgrader's picture
Offline
Last seen: 3 weeks 3 days ago
Joined: Sep 8 2005 - 9:25pm
Re: Why is Blind Testing Such a Hot Button?

I wonder the same thing. Reminds me of arguements about religions except without concern for how others feel/ believe.

nunhgrader
nunhgrader's picture
Offline
Last seen: 3 weeks 3 days ago
Joined: Sep 8 2005 - 9:25pm
Re: Why is Blind Testing Such a Hot Button?


Quote:
we don't want all magazines to look alike, do we?

Agreed. At least, this is the way I see things - the more the merrier. I'm not against blind testing but, I love subjective views. I also like JA's measurements (the shame!) and I dig solid state as well as tubes (how could he?) and guess what - I love to read about gear I will NEVER be able to afford as well (this guy is taking things too far).

I also dig flaming threads

Monty
Monty's picture
Offline
Last seen: Never ago
Joined: Sep 16 2005 - 6:55pm
Re: Why is Blind Testing Such a Hot Button?

I think cables are equally important in just about any reasonably competent system. Whether the right cable for a system is a $20 cable or a $400 cable is entirely irrelevant as to whether or not they have a substantial affect on the sound.

In many (if not most) cases, very modest systems can benefit greatly from very modestly priced cables and can be entirely unmusical with very expensive cables. As an example, let's say that a person has a budget system that has many audiophile attributes, but is a little dry in the treble. A cable designed to slightly roll-off the treble could make an otherwise annoying characteristic of the system less offensive and more musical whereas an expensive cable designed to benifit systems that are ultralinear could make the budget system completely unmusical.

To a certain degree, cables can effect very specific areas of the frequency band, though often times with trade-offs in other areas.

The fact that cables cost so much and that many manufacturers would prefer not to disclose the relatively straight-forward science behind cable design keeps this issue a matter of contention rather than an accepted area of importance to sound reproduction. This criticism does not apply to several respected manufacturers who very openly discuss the topic in their literature and websites, such as Cardas and Audioquest.

Monty
Monty's picture
Offline
Last seen: Never ago
Joined: Sep 16 2005 - 6:55pm
Re: Why is Blind Testing Such a Hot Button?

I don't think the reviewers will be inclinded to prove anything to anybody as far as what they believe to be the "best" sound or component in "their" room.

Translation: The reviewer really liked the component a whole lot and wants you to know it, just in case you are in the market for a similar component and get a chance to listen to it for yourself.

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

As usual, I'm late to the debate. Still, I would like to add an opinion. I tried to read all the entries on this topic, but my eyes began to glaze over. I think there are 2 issues in Mr. Atkinson's question. The first is obvious and openly stated in the question, and has been pretty much answered. Those in favor of blind testing appear to have a "you can't fool me" attitude. They suggest that much of the luxury premium in the prices of expensive, "deluxe," prestigious gear is due to some hoax being pulled off by elitists (one respondent used the term, "mysticism," but probably meant "mystique": mysticism involves an ESCAPE from the senses, not an indulgence such as we enjoy in music). Sparks fly and hot gases befog all when those on the other side, the hoodwinked big spenders, have to defend their position and expenditures. The second issue emerges as a sort of corollary: which is VALID, or "superior"? Not what Mr. Atkinson asked, but grist for the grinding, once the defenses proceed. I think the first issue is pretty much settled: "Hah! boy, did YOU get screwed, ponying up 20 grand for an amp that we blind testers couldn't distinguish from a $1200 integrated" vs "Did not! I KNOW my dollars were well spent and have the good taste to demand the best."
There's your hot button.
The second issue? Which is valid or superior, blind testing or sighted auditioning? Mr. Atkinson touched upon the answer, but didn't HIGHLIGHT it (or my eyes glazed over too soon...). TIME is the key. You can't stay blindfolded for six months. And it takes at LEAST that long to process all the variables and pass judgment on any well-designed piece of equipment. It took me two years to realize that my Mirage M1-si's were really, in spite of their reputation for silkiness and smoothness, quite splashy and even edgy in the 4-8 kHz region. I liked other aspects of the speakers so much that, like an initially painless sliver that begins to manifest itself slowly, the hidden irritant finally got exposed to the extent that I could no longer live with the speakers. This sort of thing NEVER gets out into the open in a blind testing situation. One has to try hundreds of different recordings, suffer hundreds of different mood changes, attend many live concerts and compare the memories, and (in spite of the efficacy of power conditioners)listen at different times of day. Mr. Atkinson's example with the Quad amplifiers HAD to take place over vast stretches of time, and THERE IS NO WAY YOU CAN DISCOVER THESE KINDS OF IMPORTANT CHARACTERISTICS IN ONE, TWO, OR EVEN TWENTY BLIND SESSIONS. I would argue, from my 40 years of experience, that these are PRECISELY the characteristics that make the difference between a vague sense of satisfaction and mere acceptance, on the one hand, and a genuine sense of excitement on the other. Validity? A moot point. Nobody can stay "blind" long enough to do the job. "Blind testing" is a red herring that TEMPTS one into being "scientifically" hard-nosed and empirical, but is finally impossible to pull off. I fully expect, though, that it will remain a hot button for as long as folks gather to try to translate music into words. That, too, is impossible. It's fun trying, though, eh?
Cheers, and happy listening to ALL of you.
Gerald Clifton

Monty
Monty's picture
Offline
Last seen: Never ago
Joined: Sep 16 2005 - 6:55pm
Re: Why is Blind Testing Such a Hot Button?

We could just pose John's question and Gerald's answer and reduce the bandwidth of this thread. Good stuff, Gerald.

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Thanks, Monty. Was it William James or James Joyce who said, "God is a noise in the street"? Your pic, which I can't love enough, suggests She comes from, er, under the street. I finally bought one of those new drivers today... you know, one that looks like a UPS truck on the end of a stick. Didn't help. Sob. Should have blind tested it. There may be a moral in here somewhere, but I'm too tanked up on Glenmorangie to pin it down. Golf and music. Hmmmm. Life is grand. I hope you, right now, are having as much fun listening to some tunes as I am. Cheers, Clifton.

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Dear Anonymous,
No, that never did occur to me because I have never had any problem keeping myself busy: there are books to read, concerts to attend, rounds of golf to play, nags to bet on, wines to drink, women to chase, new recipes to try...the list is endless. I am not much of a tweaker, really, since I know so little about what makes the lights on the boxes blink and the elves run through the wires. Music is a lifelong passion for me, not a hobby. My PhD is in Literary Theory (UC Irvine, 1978), not electronics. I enjoy live concerts so much that, in over 40 years I have probably spent close to a quarter of a million dollars on subscription concerts and the transportation to get me to a seat. That's not a lot of money, when you figure the cost of gasoline, air travel, and the actual tickets...about 5 or 6 grand per year. When I was a young pup, you had the choice between "west coast sound" (Altec, K-horns, JBL, Cerwin-Vega...so-called "PA" or "theater" speakers that were loud and efficient, but burned your ears with horrendous peaks in the upper midrange) and "east coast sound" (AR, KLH, and the like...inefficent and somewhat murky, but gloriously silky in reproducing the massed violins on orchestral recordings). Clarity and brass with irritation, or murk and silk with no irritation. Since that was all we had, we vacillated back and forth and lived with the extremes, shrugging our shoulders and saying, in effect, "if that's the best I can do, so be it...I'll let my imagination fill in the blanks." Flash forward. NOW, we have choices. There are a thousand speakers being made and sold as we speak that I could live with for the rest of my life. AFFORDABLE speaker/amp/source component systems that come quite close to the preferable center between the two extremes I just described. But with choice comes desire: we always want juuussst a wee bit more realism. You said you weren't sure all this dissatisfaction "has much to do with actually listening to music." Au contraire. It has EVERYTHING to do with listening to music. Irritants intrude. Period. You want them to go away PRECISELY for the reason that you want to "listen to music."
I have had the following components for at least 10 years, and I still enjoy them. The Adcom GFP-750 preamp, the Meridian 508-24 CD player (now used as a transport, since my new Musical Fidelity "TRI-VISTA" DAC improves the sound), the Sony SACD 777ES for SACD's, the Fanfare FT-1a tuner, the Audio Synthesis "Passion" passive preamp, the Basis 1400 turntable + Rega RB 350 tonearm + a Benz-Micro M9O cartridge (still sweet and fine, because I clean it after each LP side and vacuum the records...a pain in the ass, but grit kills cartridges and I want this one to last another 10 years). My speakers are now the Triangle Volante 260's and I love them so much I probably will die with them: they cured the irritation that finally emerged in the Mirage designs (and in the Dynaudio pair that I recently sold) with NO sonic penalties. I can't imagine changing them, being an old fart, but technology marches on...I don't think they will ever IRRITATE me, but I may hear something down the road that I like better, and make the change for POSITIVE reasons rather than negative ones. Also, I am single, heterosexual, occasionally horny (the fires gradually burn down with age, thank God, but never die), AIDS-free, gamesome, and usually in a happy mood. I honestly don't know my blood type, but I'll find out if you need to know. Oh. I still occasionally experience mild malaria attacks (a souvenir from 'Nam) but they're no worse than the 'flu. Anything else you want to know? Good. I'm outta here. Cheers and happy listening. Gerald Clifton.

Buddha
Buddha's picture
Offline
Last seen: 2 years 9 months ago
Joined: Sep 8 2005 - 10:24am
Re: Why is Blind Testing Such a Hot Button?

Mr. Clifton.

I've really enjoyed your posts here.

I wholeheartedly agree about how some negative attributes only become apparent over time, and how they can become intolerable, as well.

Anyone who's ever dated has learned this phenomenon first hand.

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Yo from the 'hood, Buddha. We gots a rainy day here in LALA land, so's I be wastin' my time on the web. Thanks so much for the kind word. First name is Gerald, but I've always preferred Clifton or Jerry, being the informal sort. Actually, I have enjoyed your posts, too, but I am so techno-illiterate that I have nothing to contribute to the "wireless music" debates. They are interesting to read, though, and eventually I'll probably learn something. Yeah. Women. The comparisons you suggest are almost too rich with innuendo to even begin to contemplate. Don't get me started or I'll start comparing tweeters to what I can remember of the gals I've chased, including the few I actually caught up with. Not woofers, mind you...I DO have a SHRED of decency. Then I'll REALLY catch hell from some of the Forum denizens. It is really fun for me to meet folks like you and Monty, because exchanging ideas with others in passionate pursuit of the perfect concert-in-the-den is sure to bear some fruit, and we all know the "Hot Buttons" fade out when the music fades in back in the listening chair. May all your tunes be clear and sweet. Clifton

ohfourohnine
ohfourohnine's picture
Offline
Last seen: Never ago
Joined: Sep 1 2005 - 7:41pm
Re: Why is Blind Testing Such a Hot Button?

Let me add my, "Welcome aboard" to those of the others. I nearly ignored your posts on the grounds that the thread had been beaten and beaten again. Good thing I didn't. Good Stuff and well put. How can an apparantly sane, thoughtful, articulate, heterosexual man with appetites for some really good stuff get along in your state full of and run by loonies? No wonder your travel budget is substantial.

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Thanks for the kind words, Cheapskate. How do I manage in a state run by a steroidally-deranged ex-weightlifter, whose best ideas come from his "glutes"? Well, the weather's good. Lots of Double-Breasted Mattress Thrashers, for those of us interested in bird watching. Hmmmm. Oh, yeah -- there's Disney Hall and lots of fine tunes from the LA Phil. And golf. Beyond that...? Well, I'm afraid you're correct. I try to steer clear of what Woody Allen called "the wheat germ serial killers." And booze helps. Next Friday, I get the Mahler 5th at Disney. Can't wait! I think Mahler is one of the ultimate tests for a good system, which brings up an often-ignored audio-truth (for me): if I enjoyed only small ensembles (Jazz, Pop, and/or Classical), finding a cheap system nowadays would be easy. But, damn, it's hard to get even a semblance of a full orchestra on record or CD, into an amplifier, and out your speakers. I have heard many systems that sound great with solo vocalists and instruments and small ensembles, but you throw in some Mahler, Berlioz, or Tchaikovsky and the edifice starts to crumble with almost all of them. On the other hand, if you can find a system that gets a reasonable facsimile of the entire orchestra into your room, it will always sound fine on everything else. At least, that's been my experience. Makes sense, I guess, because the illusion required is so much more stretched with 150 musicians, which you couldn't even get into your yard, than it is with a guitar trio, which will fit into a small cocktail lounge. Still, many system/component reviewers will rave on about the equipment without testing its breaking point with, say, the Mahler 2nd, Berlioz Requiem, or Tchaikovsky 5th. I'm not saying classical is SUPERIOR (although I prefer it), since tastes are personal, but that a reviewer should go all the way with the most demanding software when evaluating, not leave the job only half done. There are a couple in The Absolute Sound who only play Patricia Barber and Diana Krall and go into ecstasy over the sound. I just hate that. Cheers, and keep the good thoughts comin'...Clifton.

Jeff Wong
Jeff Wong's picture
Offline
Last seen: 3 years 3 months ago
Joined: Sep 6 2005 - 3:28am
Re: Why is Blind Testing Such a Hot Button?

I just wanted to pop in and mention I've enjoyed reading your posts as well.

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Thanks, Jeff. I hope some of you are going to HE '06 and I can get a chance to buy you all a cold one. This is the first one in quite a while out here on the left (lost?) coast, as the one in San Francisco was cancelled (the labor dispute). Happy tunes! Clifton

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Yes, yes, yes. I have only been to a couple of these. The first one I attended was in the early 1980's, when Dahlquists and the Infinity "Servo-Statics" were the talk of the show. It was VERY early on in Stereophile's attempts to put on a show...hell, it may even have been the first one. It was in a Holiday Inn-type place near the airport, with only one floor and about a dozen or so exhibitors, as I recall. The last one I went to was at a large airport hotel (the Hilton??) in the late '90's. And you are right: it seemed that 95% of the exhibitors played soft rock, pop, and jazz (LOTS of jazz!). The folks at Totem were the nicest. They had the Mani-2's set up in the windows (!!), like air conditioners, their butts half hanging out into the salty breeze. They sounded TERRIFIC! They had a Thorens turntable jerry-rigged on top of an end table and interrupted Patricia Barber's "Modern Cool" (God! I wish I had a buck for every time I heard THAT at the show...) to throw on my Ormandy LP with the Philadelphia doing the Berlioz Requiem. 4 sides! And LOUD. People slowly started leaving, during the Deus Irae. One of the Totem guys said "good riddance" -- turned out he was a Berlioz freak. I couldn't have been more impressed with the sound from those tiny boxes. Bass to die for! But they were quite insensitive (around 86 db, as I recall) and at the time I was itching to get back to 25-watt tubes. I may yet end up with them -- I hear they have a new "Signature" upgrade just recently out. I met Dennis Had (the designer for Cary) outside in the beer garden, and we had a few beers and a couple cigarettes: there were NHT outdoor speakers doing the background music. It was "new age" -- sounded like trickling piss. Dennis was a great guy to talk to. I hope the one this June is as much fun. Cheers and thanks for the comments. Clifton

dcrowe
dcrowe's picture
Offline
Last seen: Never ago
Joined: Sep 1 2005 - 9:39am
Re: Why is Blind Testing Such a Hot Button?

Thanks, Anonymous. Archiving the ancient history of this forum may have some value (amusement value?) some day.

ohfourohnine
ohfourohnine's picture
Offline
Last seen: Never ago
Joined: Sep 1 2005 - 7:41pm
Re: Why is Blind Testing Such a Hot Button?

"If we can hear it, and it's real, it must be measurable by instrumentation in some way, because our ears have already 'measured' it. Whether we have the equipment to measure it is another question. We do have the means to hear it.

Finally, the improvements to the system has allowed me to get even closer to the musical performance. Solo violin has a remarkably detailed presentation now. I enjoy listening to music more."

Fascinating. You've attempted and achieved an improvement in the sound of your system borne out conclusively by listening tests. That isn't enough. You want a scope to agree with you. You want a graphical presentation and some numbers and indices - perhaps as yet unnamed.

How much do you love your wife or your dog or whatever it is you love? Do you crave a numerical index for that? Should we assign a number to the differences we hear when Heifetz and Perlman both play the Beethoven Concerto? How about numbers for Monet and Renoir? Why not leave to science what is science's and allow art some place in our lives?

I think you hit the mark in your final sentence - you're enjoying the music more. Wasn't that the point?

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Dear anonymous,
Thanks for the comments. Yes, that is what I meant...your paraphrase is accurate. The SBT "experiment" you describe is interesting. I think many of us have done similar "tests" casually, even in a dealer's showroom. A salesperson will pop in while you are auditioning, say, speakers, and he will fiddle with a bewildering array of amplifiers on display; then the sound will change (without your having any knowledge what he really did), and he will say, "What do you think of that?". Usually, it sounds better and costs more, or the salesperson wouldn't have bothered. Remember when you could press a button on a gizmo and different amp/speaker/source combinations would switch back and forth? After about 3 switches, you were wondering if the differences were significant or imaginary. Auditory confusion would set in, especially after the differences started to become quite subtle. The first thing I would do in such situations is work back to the gross differences I was confident in identifying and retaining (i.e., speakers that were peaky in the highs vs. smoother ones)...trying to establish a reference I could easily identify. Usually, I would walk out shaking my head, without purchasing. "I'll try again tomorrow." I think that's why audio salons stopped, for the most part, using such demonstrations -- they only confused the customer, and confusion never sells. It boils down to gross differences vs. subtle ones, and, as you mention, one has to live with the subtle ones for quite awhile before one becomes confident enough to act and make changes. I think you probably do have "golden ears," because you enjoy music and are sensitive to different listening situations. I think "golden ears" is a catchall phrase that appeals to our desire for certainty in establishing references: we want an objective control "out there," against which we can confidently make decisions about the music we love. Sort of like Dorothy wanting a real Wizard.I don't think such certainty is possible, because our listening experiences (both live and recorded) change so much over time. And, in the long run, it is after all our own subjectivity that we have to live with. SBT and DBT, to me, are attempts to "get outside" ourselves and access some objective truth. We want that certainty. Such experiences always SEEM valid if the differences are gross enough to give us confidence in some fixed, stable standard. But it's those damned subtleties that eventually, over time, drive us nuts...the nuances we can PERCEIVE, but, alas, not CALIBRATE. Thanks for your interesting post. Register for the forum! You can learn more about power-listening from DUP (sorry -- inside joke). At any rate, cheers and happy tunes! Clifton

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Anonymous,
First, you don't have to worry about keeping up with every response to your posts. We're a casual bunch and certainly understand that almost everyone works for a living and can't be on the web 24/7. I predict you will hear quite significant differences with VTA adjustments. When I bought a new cartridge from Ambrosia Audio a few years back (a $1200 Benz-Micro) a wonderful salesman named Andy set it up on my TT free. He really knows what he's doing -- he started with the manufacturer's recommendation, which turned out to be imprecise, but okay, and tweaked by ear from there. I listened in. At the manufacturer's setting, the sound was smooth and spacious, but there was a slight midrange suckout. After he fussed around a bit, the midrange became noticeably more neutral. But that's not the half of it. I hadn't noticed, but before the final adjustments, the sound had been "2-storied," for want of a better term. All the treble was "up there," above an imaginary line, and all the lower midrange and bass was "down below." After Andy tweaked it, the sound snapped into a coherent unity, much more like the coherent sound you hear at a live concert than before. I couldn't believe my ears.
I used to live in "The OC." I just got off the 'phone this morning after purchasing season tickets at the Segerstrom. Check it out if you haven't already -- Maazel and the NY Phil, Gergiev and the Kirov, Eschenbach and the Philadelphia, Andrew Davis and the Pittsburgh, Labadie and Les Violins Du Roy (doing the Mozart Requiem!)...the list goes on. Cheers and happy tunes. Clifton

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Yes, analog setup can be tricky -- there is a "juuusst right" point for every parameter. Fortunately, you can hear the differences between, say, too much VTA and too little, and you can get it right by ear, especially if you have a friend to help you out. I don't remember analog being this fussy back in the "old days," when names like Garrard, Dual, Weathers, Shure, and Pickering dominated the scene. This is probably because today's gear is so much more advanced and thus dependent on finer tolerances. Cheers, Clifton.

commsysman
commsysman's picture
Offline
Last seen: 1 day 3 hours ago
Joined: Apr 4 2006 - 11:33am
Re: Why is Blind Testing Such a Hot Button?

It's like religion; everyone is sure they are right, and do not want to be confused by anything that conflicts with their dogma.

Personally, all the arguments I have seen, pro and con, miss the main point as far as I am concerned.

When I evaluate a system or a component, I can hear significant differences very quickly IF I am listening to specific passages from specific recordings that I have carefully chosen because I find certain characteristics of those recordings very revealing of certain differences.

I have five tracks from four discs that I use (Beethoven Mass in C, Kyrie; two tracks from "Cantate Domino"; Poet and Peasant overture (Telarc); and Old Friends (Alte Kamaraden) from Hands Across the Sea (Mercury). I am so familiar with these, and what I am listening for in them, that I hear differences quickly and fairly accurately, I think. This is not where I stop, of course, as I have at least a dozen other recordings I use initially, but these 5 tracks will give me a very good first impression of the qualities of a system and tell me quite a lot about it.

To do blind testing with material that the listener is NOT intimately familiar with in the first place is ridiculous; it eliminates any possibilty of any meaningful results, unless the system differences are SO colossal that there is no point in a comparison anyway (a boom box against a class A $100K system).

HOW CAN YOU USE THE UNFAMILIAR AS A REFERENCE TO EVALUATE DIFFERENCES IN OTHER UNFAMILIAR THINGS???

I think blind testing can work, but only if the subjects are given the music to be used well in advance, with instructions to listen to this material many times on a familiar system, with particular attention to lets say the harp in the background at the middle of this passage, or the attack and decay and general dynamics of the bass drum at the start of this passage, the quality of the massed voices or strings here, the piano there, etc.

I have NEVER heard anyone stress the importance of this kind of protocol in a blind test, and to me this would be the most important protocol of all; absolutely essential.

I would ridicule any blind test of anything that did not include and stress it.

Trying to sort out any equipment difference when listening to material the listener is not intimately familiar with is, in my opinion, like trying to get definitive opinions on some fine wines when none of the participants has ever tasted wine before.

Lotsa luck!!!

Buddha
Buddha's picture
Offline
Last seen: 2 years 9 months ago
Joined: Sep 8 2005 - 10:24am
Re: Why is Blind Testing Such a Hot Button?

Commsysman, I had missed your post. It was terrific.

I've been thinking a little about this "being on the spot creates a tendency toward null results" stuff.

Why doesn't sighted reviewing cause a trend toward null results, as well?

If DBT does it, souldn't any listening "under pressure" tend to push toward homogenization of the listening experience?

Why does sighted reviewing yield such "order of magnitude" and "night and day" proclamations but DBT does just the opposite?

In both situations, we are putting the listener to the test, no?

Just having diarrhea of the keyboard tonight.

How about this DBT:

Get a preamp or CD player with a polarity switch and try a few sessions with people where nothing in the system changes except the position of the polarity switch and see if you get any results.

No extraneous devices required, just seeing if people could pick up on that.

Exploring DBT should be called for more than blanket endorsement or condemnation.

How about we see at what point or situation people do become able to pass DBT tests?

If we could demonstrate just how big a difference has to be in a certain parameter before people notice, we could learn from that.

Cheers.

Yiangos
Yiangos's picture
Offline
Last seen: 2 years 10 months ago
Joined: Sep 7 2005 - 8:41am
Re: Why is Blind Testing Such a Hot Button?

And no one even mentioned a DBT group test yet ! ROTFL

bengrbm
bengrbm's picture
Offline
Last seen: Never ago
Joined: Mar 28 2006 - 10:33pm
Re: Why is Blind Testing Such a Hot Button?

Hi Buddha -

Great posts. I think its always worth remembering, as you seem to, that most 'phile readers don't want a purely "objectivist" or "subjectivist" mag in any pure sense. We like the combo of subjective reviews and measurements, and would hate to lose either. I personally: 1) Don't have a problem w/ DBT when its done well and think it can be informative, 2) Would love to see it as a component of reviews when possible, and 3) Understand that for some equipment it may not be practical due to time and budgetary constraints. I think some of the sceanarios you and commsysman suggest would be fun to read and the magazine writers could provide the proper context/discussion. I don't really get how this has turned into such a heated debate...

Ben

tandy
tandy's picture
Offline
Last seen: Never ago
Joined: May 24 2006 - 3:57pm
Re: Why is Blind Testing Such a Hot Button?

Right on. Sometime, investigate one and count all the assumptions made. Even the administrator won't call them fact, and the administrator usually has alot of training and experience with other DBT tests. But one will often see some hardcore screwball on the forums excercising extreme license by calling them "fact".

---------

IN response to the above post. Maybe that is part of the problem, the assummptions etc that are made in dbt testing.


Quote:
Having read about 1/3 of the posts, I think Clifton's second point is well-put, and as he says, the first one is answered.

This is my very first post to this forum. As an introduction, I am trained in the sciences (both physical and life) and engineering (physical and biologic). I also have amateur experience in two musical instruments, and also fortunate to have 'absolute pitch' hearing ability, which has immensely helped in the audio hobby. I do NOT think I have "golden ears". I do not have the time (I wish I did) to fiddle with audio equipment as I had before.

In short-term vs. long-term listening, blind or not, there are degrees of differences that may be audible or not depending upon the situation in which the listening is conducted (a hypothesis). This is my interpretation of Clifton's description of requiring long listening to finally let the little annoyances get to you enough to take action, that would go unnoticed in short listenings.

As a musician, I (and others) can hear the difference between two pianos, or two violins, or two guitars, etc. Why is there doubt we can do the same with audio equipment? Can hearing be decieved? I've read that it can, but I'm no expert in this.

On the other hand, I have tried to do some DBT. It is difficult to do correctly, and I do not think I have yet done so correctly. But I do think that it's a valid method to test differences, but maybe not all the kinds of differences that occur in audio equipment (conjecture).

Anectdote:

I conducted a SBT with the help of a music teacher. He listened while I made a single change to my equipment, back and forth between two states, call it "A" and "B". He did not know what the change was, or whether I made the change or not. He could see me while making the change, but could not see what the change was, if any. His description of "A" was there's more treble, but the sound is bunched-up toward the middle. For "B" there is less treble but the sound is spread out wider horizontally. He was 100% correct in discerning whether I used the A, or the B modification.

Was the above a controlled test? No. Was it DBT? No. Was it clealy audible? In my opinion yes. I also heard the same kind of difference, and wanted to see if someone else could also.

Another experience is with the capacitor and wire modifications, e.g. polystyrene/polypropylene caps or teflon-insulated wires, instead of electrolytic and PCV, respectively. As a traditionally-trained engineer, the cap and wiring mods did not make sense that it should affect sonics. Having replaced the 'bad' caps with 'better' caps and the wiring with teflon-insulated ones, I was surprised to hear that it did make a difference for the better. What I heard was that I could focus on an individual instrument and follow it throughout the song regardless of other instruments playing. Each instrument was 'trackable'. Before the capacitor mods, I could not, as the instruments seemed to interfere with each other, i.e. when one instrument got louder than the one I was trying to 'track', the louder one obscured the one I was focused on. The result of the mods is more like the live musical experience than the one before the mods. Of course, I preferred the modified system to the un-modified, and wondered why manufacturers don't already do this.

No DBT was done because I don't have two systems, modified and un-modified, to compare. Do I hear the difference? Yes, and I can describe it. What causes the difference? I don't know, but the mods are probably responsible. (keep in mind that not only were the caps and wires changed, the de-solder and re-solder is part of the mods. The wire location is slightly different. The interconnects were un-plugged and re-plugged.) What to measure? I wish I had the equipment to do so.

End of Anectdotes.

I like to see more discussion of how we can use DBT to correlate to listening experiences. If we can hear it, and it's real, it must be measurable by instrumentation in some way, because our ears have already 'measured' it. Whether we have the equipment to measure it is another question. We do have the means to hear it.

Finally, the improvements to the system has allowed me to get even closer to the musical performance. Solo violin has a remarkably detailed presentation now. I enjoy listening to music more.

Hope the information helps.

bengrbm
bengrbm's picture
Offline
Last seen: Never ago
Joined: Mar 28 2006 - 10:33pm
Re: Why is Blind Testing Such a Hot Button?

What are those assumptions, exactly? Are they more sever than the assumptions in a cited test? Like assuming that knowing who the manfacturer is won't severly condition the outcome even if they are, say, an advertisor?

That seems like serving two masters to me...

Monty
Monty's picture
Offline
Last seen: Never ago
Joined: Sep 16 2005 - 6:55pm
Re: Why is Blind Testing Such a Hot Button?

I've seen Stereophile call it like they see it for a lot of years. And, I've seen more than a handful of far less than positive reviews of equipment with big names on the badge to have little reason to distrust Stereophile under JA's leadership. I wouldn't say that about any other audio magazine, including TAS.

While I am not an advocate for DBT, I do think they could be fun and interesting if they were approached from a fun and interesting perspective, rather than an attempt to feed the fire of controversy. I think most people just want to see the reviwers measured. To me, this is like thumbing your nose at decades of experience, devotion and education about audio reproduction in search of an exciting train wreck.

tandy
tandy's picture
Offline
Last seen: Never ago
Joined: May 24 2006 - 3:57pm
Re: Why is Blind Testing Such a Hot Button?

"What are those assumptions, exactly? Are they more sever than the assumptions in a cited test? Like assuming that knowing who the manfacturer is won't severly condition the outcome even if they are, say, an advertisor?"

Show us a subjective audio test as fact, if you can.

bengrbm
bengrbm's picture
Offline
Last seen: Never ago
Joined: Mar 28 2006 - 10:33pm
Re: Why is Blind Testing Such a Hot Button?

I pretty much agree with you - I wasn't implying anything about stereophile, just responding to a post. As for 301 - is it possible for you to answer a question without posing another one...

tandy
tandy's picture
Offline
Last seen: Never ago
Joined: May 24 2006 - 3:57pm
Re: Why is Blind Testing Such a Hot Button?

"I pretty much agree with you - I wasn't implying anything about stereophile, just responding to a post."

Ok, fine. Glad you cleared that up.

greenelec
greenelec's picture
Offline
Last seen: 3 months 2 days ago
Joined: Feb 10 2006 - 12:37am
Re: Why is Blind Testing Such a Hot Button?

Albert Einstien, "Not everything that counts can be counted, not everything that can be counted counts."

absolutepitch
absolutepitch's picture
Offline
Last seen: 8 months 2 weeks ago
Joined: Jul 9 2006 - 8:58pm
Re: Why is Blind Testing Such a Hot Button?

Just a kind reply to Clifton, and others, who encouraged me to register. I was the "Anonymous" one that posted #6161, #6255, #6259 and #6640. I decided to join the club, even though I only get to read and contribute sparingly.

I have not had any time to do further investigations in a SBT mode. Nothing new to say about DBT except that I would still like to see it correlate to listening, especially in cause and effect.

I would specifically like to see an experiment of the following sort. Get two, identical brand/model pieces of equipment. Somehow demonstrate apriori that these two are 'identical' pieces of equipment. Then, modify one of the two. Perform measurements and listening tests (long-term as well as DBT). Are they then distinguishable from each other and in what way (how do they differ in measurements and listening)?

gkc
gkc's picture
Offline
Last seen: Never ago
Joined: Feb 24 2006 - 11:51am
Re: Why is Blind Testing Such a Hot Button?

Welcome to the forum, WTL. As indifferent as I am to DBT, it would be interesting to me if the modifications were of the sort pretending to improve the equipment. Maybe this was what you meant. So many "after market" shops have sprung up lately that "mods" have become a significant industry. I would like to know which ones actually succeed and why. Commsysman has posted some great comments about his experiences with updated caps on his preamps, as well as other improvements wrought by Audio Research. Since these aren't exactly mods, but meant to be evolutionary in their line, I would be talking more about shops that are not affiliated with the originating manufacturer. I can remember Art Dudley remarking that he was going to get his Sony SCD-777ES modified, and comments were to follow, but I either missed the follow-up or he decided not to go ahead with it. I suspect the modifications business, like the manufacturing business, has its own mix of the competent and incompetent, and I wish more reviewers would comment on their experiences, positive and negative, with this growing sector. This seems made to order for DBT, since many mods do not change the cosmetic appearances of the components. Cheers, Clifton

absolutepitch
absolutepitch's picture
Offline
Last seen: 8 months 2 weeks ago
Joined: Jul 9 2006 - 8:58pm
Re: Why is Blind Testing Such a Hot Button?


Quote:
As indifferent as I am to DBT, it would be interesting to me if the modifications were of the sort pretending to improve the equipment. Maybe this was what you meant.

Yes, Cliftion, I was referring to mods that are claimed to improve the sound, as heard. In the prior posts, I mentioned capacitor mods, the VTA of the cartridge, as well as interconnects of PVC vs. teflon, and others. The interconnects can be evaluated back and forth switching between the two. But cap mods can only be tried with two pieces of gear, i.e. two pre-amps, or two amps, or the like.

Mechanical damping has been suggested to greatly improve the sound of modestly-priced CD players. Although this falls into the same group as other electronics where typically a person only owns one unit and not two, it might be possible to make the mod while listening to the unit, say through headphones. I DO NOT suggest to do this with the cover off unless he knows what he's doing and properly protected, especially if there is a laser diode in operation, or dangerous voltages present. If the improvement is as dramatic as claimed by sequential subjective listening, then it may be possible to verify that this happens with mods installed on-the-listening-fly. Such a mod can be applied and removed to test its audibility, and amenable to SBT.

I have tried the "Rain-X" on CDs, with a dramatic reduction in treble shrillness on a Brandenburg Concerti CD. I followed-up with two identical CDs of guitar music, one treated and one not. With only one sample of twin-CDs, I heard no difference, except for a very subtle reduction of noise floor on the treated disk, tested by listening sequentially to each CD.

Again, thanks for your thoughts and encouragement to join.

Pages

  • X
    Enter your Stereophile.com username.
    Enter the password that accompanies your username.
    Loading