Who Watches the Watchers?

When we launched Stereophile's website at the end of 1997, we decided that we would not reprint the magazine's most popular features, including the biannual "Recommended Components" listings and Michael Fremer's monthly "Analog Corner" column. We were concerned that doing so would cannibalize magazine sales. As it turned out, we were wrong—and so the latest "Recommended Components" is available on our free-access website day-and-date with the publication of the April and October issues in which it appears. And starting with Mikey's very first "Analog Corner," from July 1995, I have been posting his column on our AnalogPlanet.com website.

At the time of writing, I had just finished HTML-encoding Michael's October 2004 column, and fell over an exchange between him and popular technology writer Walter Mossberg, at that time a contributor to the Wall Street Journal. The essence of Mossberg's argument was that people who say that there is better sound than listening to MP3s are snobs—that most of his readers listened to music on computers through plastic speakers and "love—love" listening that way.

Michael responded that all people had to do was to listen for themselves. This, of course, was the rock on which Stereophile was founded when the late J. Gordon Holt published his first issue, in November 1962. And Gordon presaged Mossberg's point by almost two decades when he wrote in 1987: "As the person who 'invented' subjective testing, I have followed with great interest the many articles in the mainstream audio press which purport to prove that none of us can really hear all the differences we claim to hear, particularly those between amplifiers."

Differences between amplifiers? I flashed back on an amplifier listening test in which I was involved in 1978. Before becoming an audio journalist, I'd been a professional musician, and before that I worked in scientific research. My formal education had been in the sciences, both at the bachelor and postgraduate levels, and while I wasn't a "hardcore" objectivist back then, I was skeptical of those who wrote about enormous differences in sound quality. Martin Colloms had organized a test for Hi-Fi News magazine in which we would try to identify three amplifiers, including a solid-state Quad and a tubed Michaelson & Austin, by listening to them under blind conditions. Martin's test methodology seemed, to this erstwhile scientist, to be beyond reproach, and when the analysis of the test results showed that there was no statistically significant audible difference between the amplifiers, I bought a Quad 405.

Worst audio purchase I ever made. I stopped getting any musical satisfaction from my system. It wasn't until I replaced the Quad with an EL34-tube–equipped Michaelson & Austin TVA-10 that my musical enjoyment returned to its previous level.

All I could conclude was that the blind-test protocol itself had become what scientists call an "interfering variable"—that the conditions of the test were too far removed from how we listen to music through our systems to give meaningful results. I have participated in a large number of blind tests since 1978, and have found that, even when a real difference exists, it is very difficult to produce anything but statistically null results—what are called "false negatives." In the end, I decided that if so-called "objective" testing lets you down, it's best to follow J. Gordon Holt's strategy and judge equipment by how it sounds playing real music in real time.

This has pitfalls of its own. When such tests are poorly performed, they can produce "false positives"—ie, the listener concludes that there is a difference when none exists. In an article in the November 1980 issue of Hi-Fi News & Record Review, since republished on this website, I wrote: "The problem confronting the magazine reviewer when organizing the necessary listening tests to accompany/reinforce the measured behavior of a device under test is complex. . . . Unlike the reaction of an oscilloscope, that of a listener involves interaction: what he is hearing; what he had been expecting to hear; the identity of the equipment; the emotional effect of the music program; the emotional effect of other competing stimuli (a recent cup of coffee, a not-so-recent visit to the toilet); the apparent expectations of his fellow listeners; the ultimate purpose of the test; the desire for self-consistency and hence self-esteem; all these can—but needn't always—color the listener's assessment. Obviously, this will affect the reliability of any conclusion, both when used to predict the same listener's reaction to the same piece of equipment, and when used to predict other people's reactions."

I read enough reviews published elsewhere where I do indeed doubt the reliability of the writer's conclusions. As Stereophile's editor, therefore, I work hard to ensure that, when you read our reviewers' descriptions of sound quality, those descriptions are both repeatable and transportable. We include enough information in our reviews about the music that was used in the testing that you, the reader, can use the same music to see if you agree with our findings. When possible, I visit the members of our team and listen to their systems, where inserting the product being reviewed is the only change. That listening may seem informal and collegial—as it was with the Revel Salon2 speakers Jim Austin writes about in this issue—but it is basically a test of the tester. I want to be able to hear for myself what the reviewer describes. And when a reviewer's auditioning of a product raises questions, I listen to it myself in my own system.

Stereophile's readers deserve no less.

COMMENTS
tonykaz's picture

I've worked in the HighEnd Audio Industry and the Automotive Industry.

People, by and large, strongly prefer the non-intrusive character of MP3 ( Car audio ) type Music Playback Systems to our ( my ) HighEnd Audio Systems . The difference in popularity is probably 99.98% vs .02% annnnnnndddd I can prove it! ( but all of us already know it fully well )

There certainly are people that like Big Sound from time to time ( swimming pool parties ) but it's an exception. People will actually shun an Audio System owner playing ( showing-off ) the Audiophile System, it's part of the reason that Magnepans are more accepted in the main living spaces of a home ( rather than powerfully dynamic Klisphorns )

We dedicated Audiophiles can find ourselves banished to the Basement. ( like Analog Turntable people with giant Amplifiers, giant Imposing colossus Loudspeakers, multi-thousand 33.3 musty record collections and all the exaggerated paraphernalia we are so fond of discovering )

Sensible families might tolerate a elegantly simple all inclusive Linn 530 System that plays any source of music ever devised yet is carefully engineered for anyone to easily access.

People CAN tell the difference between MP3, they've learned that they prefer it, they will actually spend BIG money making it tiny and invisible so that they can live with it being non-invasive & non-impactful in their lives.

Dammit, I'm still a Koetsu Lover, they can't take that away from me!

-------------------------------

Quad 405

Egads,
Years ago, I was rather shocked to read the JA confession of owning the 405. I owned one too, because of it's striking front heatsink appearance, dam good price from the UK distributor, nice color chassis, multi-power supply ability and mostly because I was selling two to three pairs of Quad 63's per week. The little Amp worked and sold. I then thought that our JA was an Organization Man not an articulate HighEnd Audiophile. I still admired JA because HFN&RR was so dam Good! Then JA purchased Krell and all was forgiven. phew, close one. I'd like to suggest that every one of us have purchased a goodly bit of rubbish product...

which is why...

We need and love Stereophile.

Which is why we desperately need accurate reviewing instead of Company's Advocate's recommended Opinions.

I'll contend that we are only partly to blame for buying the wrong gear when we read over-reaching statements about marginal products reviewed for large advertisers. ( General Motors for example )

When I was B&K Importing UK gear, Manufacturing Audio Bits, Retailing HighEnd Audio at Esoteric Audio and doing our own Jury testing of Product for Sale on our Sales Floor, we were disappointed by some/much Manufactured Product being sent to us for Retail Sales, gear that was "properly" reviewed by credentialed reviewers ! Gordon Holt's criticism of the reviewing industry was accurate.

Kathy at Thiel & Quincy Jones recommended Electrocompaniet from Karen Sumner --- Superb Product.

John Atkinson recommended Koetsu --- Greatness

Enid Lundley recommended Monster Cable --- best product Line we had

Tony Cordesman recommended Conrad-Johnson --- we bought in deep, it was horrible

Karen Sumner ( now Transparent ) brought us Bruce Brisson --- Best Cable ever!!

England brought us Linn, Rogers, Pro-Ac, scads of Turntables --- all the Brit. stuff sold & performed well

the Absolute Sound recommended EAR 509 Amps which were horrible. --- Phew.

So, when reviewers resort to one of those standard closing cliches like : "should be on the short list" or "I highly recommend" or "if budget permits" ....

--------------------------------------------

Psychoacoustics

I went to a Headphone Meet at Overture Audio in Ann Arbor ( 2015 ) and there learned :

1.) that my 7 Decade old hearing tapers off above 8k which makes auditioning DACs difficult to impossible.

2.) Tubes are lovely singers , Solid State can be nice too but not glorious like tubes can be.

I sought Professional help at University of Michigan's Audiology's department. I learned more than I'd hoped to :

No.1) Music creates dopamine in our nervous system

No.2) Our personal nervous system ( ears, all connecting nerves and brain ) has been in a constant state of self tuning and adjusting ( since birth ). It can compensate for missing bits of data. It can extrapolate.

No.3) My personal hearing system prefers dynamic drivers ( sennheiser HD headphone drivers ), class A amplification and is addicted to the dopamine that tube gear triggers in my brain.

So, our psychoacoustics profile is a factor in our choosing gear, we should rely on our listening experience. ( not reviewers )

Of course, I love reading the interesting stories that other Audiophiles tell about their Adventures, it's probably all that Stereophile Writers need do well is tell fascinating stories. That Quad 405/EL34 Story is a great example!

--------------------------

I'm moving to Florida and disconnecting from all Internet.

I'm off-the-air for the rest of the year, probably.

There are wonderful things happening here at Stereophile , it's a great place and has been a lovely hibernation companion. I continue my Print Subscription and will contribute my previously loved issues to my local Barber Shop!!!

I'm also continuing Patreon for Steve G. ( what would this World be like without Steve G & HR ? )

Tony heading to Venice Florida, the Tropics, "God's waiting Room"

Jim Austin's picture
Tony, perhaps you'll weaken from time to time and find yourself on the Internet. If you do, return and let us know how you're doing. Near-native Floridian here--moved there when I was 4 or 5--not sure. Loved it then (and for the next 15 years). It changed. I miss what it was. jca
tonykaz's picture

Thanks for writing,

I'll be back ( on the Air ) as soon as I set up my next Office in Florida which I anticipate will be Jan.2020.

I have a busy commitment schedule with Bernie Florida Campaign combined with settling down and trying to retire.

Right now seems like the best of times for HighEnd Audio folks like us. I've had an involvement since Mono Mac 45 watt Amps and Preamps with adjustable RIAA switches transitioning from little 45s and 78s. It's been exciting.

PS Audio is becoming the Giant of our World, with Schiit coming-up from a nearly nowhere half/garage 2/56 screw hoarding outfit . It's almost like Linn Naim excitement of the early 1980s England. It makes me want to buy a LP12 from Overture Audio in AnnArbor. ( or maybe a vintage one with an afromosia plinth )

You Stereophile lads are raising the bar.

I wish you a Bon Voyage.

Tony

Ortofan's picture

... you'll also need to acquire a vintage Keith Monks tonearm - the one with the mercury contacts.
To complete the ensemble, you'll also need one of the "top-flight" cartridges recommended by KMAL, such as an ADC XLM. Recall that the XLM was part of the original (circa 1973) TAS reference system.
While you're at it, get a Harman-Kardon Citation 11a pre-amp, a Phase Linear 700 power amp and a double set of Advent speakers.

tonykaz's picture

I'm probably not as Vintage as all that.

I was selling LP12s in the early 1980s, just before and during the Valhalla introduction. I might want something like that, with a fresh looking Silver Ittok and visually complete Asak with a cute little alcoholic Red Nose. I'd try to cosmetically perfect it with a Clear Dust Cover and then simply have it as a Musiem Piece.

Seems rather extravagant to do if I'm not intending on it being part of a System but I've gone RedBook & Active Loudspeakers.

I'll be chasing technological advances and forgoing Ultimate Analog Sound Qualities, trying to make it easy for music to be my everywhere, everyday companion.

Thank you for offering helpful advice.

I just now came to know Tom Okeefe ( thanks to JA visiting the Quad 57 Room at Axpona 2019 ), Linn LP12 set-up man at Overture Audio, who seems able to transform these Tables into STOA Front Ends but even so, I like them the way they came from the Factory, felt pad, single speed and impossible on loose wooden floors.

Now,
I'm thinking that a person can build a 33.3 Music System
or
A person can build a Digital System

It's one or the other, they are completely different, tuned differently, playing differing Source structures, benefiting from significantly differing care and gear selections.

My dynamic transducers, Class A Amplification, Brain, nervous system and ears all have accepted and prefer the more relaxed digital format, I've morphed from Koetsu to RedBook. ( successfully ) Lucky me.

Tony in Michigan

ps. I hope that we can continue to stabilize on 16/44.1 thru 24/192 and not figure out some new HORRIBLE sounding format that obsoletes all our systems.

AJ's picture

Whereas we don't have much snow in Florida, we *do* have the internet.
It's how we get all the Florida News of the weird

cheers

AJ
Soundfield Audio

tonykaz's picture

Hello,

I'm doing a relocation, I won't have any hobby time for 6 months.

I'll miss y'all

Tony in Michigan

ok's picture

a close relative of mine underwent cornea transplantation surgery and never again perceived colors the same way both eyes: donor's transplant obviously rendered things a tad bleached-out, but there was no way for one to realize or communicate such a peculiarity save for this accidental postmortem direct comparison. Common language cannot express one’s actual experience, much less in audio where all descriptions are inescapably metaphorical for a reason. So next time someone raves or laughs about lively colors or fancy sounds –please keep your own eyes and ears wide open and go find out for yourself.

hollowman's picture

I think many of us continue to follow Stereophile because of JA's presence. And AnalogPlanet, because of Micheal Fremer.
Inner Fidelity no longer holds interest after Tyll's departure.
And AudioStream was always a confusing project (hint: digital/computer gear should be on Stereophile)

Ali's picture

Agreed

Glotz's picture

Because the two have a closer, more symbiotic relationship.

Sorry to hear Tyll left... Trusted ears and skillful writing will elevate it again.

tonykaz's picture

Way back in the Day, Ivor at Linn convinced us that : "Garbage in gives Garbage out" !

Turntable comes first,
Arm comes Second
PU third
Electronics forth
Loudspeakers fifth.
Wire never
Spiked Stands against the Wall
All the above was ideal for vinyl 33.3 in Tiny Homes that typical Brits live in. ( with spiky, horrible, noisy, 220 Mains Voltages )

It still seems like a logical way of building a lasting and satisfying Music System, especially if on a limited budget and having to co-operate with a con$ervative Banker.

HOWEVER ---

Today,
Red-Book digital is pretty dam good, players are pretty dam good, electronics are mostly tending to be pretty dam good making the Loudspeaker choice the "ideal" way to begin building a Music system. Paul McGowan preaches effectively on this concept.

So, 33.3 & RedBook probably need vastly different Philosophical approaches to Equipment Ownership.

33.3 needs a multi-thousands of dollars commitment in Front End to start! A New LP12 for around $10,000 or more for a New VPI

An Audiophile Grade RedBook system could get a good start with any of the Loudspeakers that Steve G., HR, JCA suggest. All for well under $10,000!

MP3/RedBook is "way-less" pricy than a proper 33.3 System.

We 33.3 folks are facing a difficult financial Curse, is it worth the extravagant expense & effort?

Tony in Michigan

ps. we can still find plenty of record players at garage sales, thrift stores and estate sales ( well under $100, NO Refunds!!! )

ps.2 It's pretty hard to find anyone that still mostly plays their Record Player if they own a Streaming System, isn't it? and most curious Audiophiles already own and listen to Streaming.

ps.3 It just seems like our 33.3 guys are becoming our Curators. ( god bless em )

Long-time listener's picture

"All I could conclude was that the blind-test protocol itself had become what scientists call an 'interfering variable'—that the conditions of the test were too far removed from how we listen to music through our systems to give meaningful results."

YES YES YES YES YES ! .....

"And when a reviewer's auditioning of a product raises questions, I listen to it myself in my own system."

And what do you do when one of your own reviews raises a question? Your review of the Aerial 5T raised a lot of questions in my mind. I found it to be seriously lacking in bass response, with thin and colorless sound overall. Yet you gave it a Class A rating, even while noting you had to use it very differently than the manufacturer recommended. More second opinions would be VERY worthwhile. (Which is not to say that some JA reviews have not been very helpful. The NAD M32 one a case in point.)

Stereophile readers deserve no less.

jimtavegia's picture

Most of the folks who MIGHT think that MP3 is just fine would probably not drink the cheapest coffee on the planet knowing full well there IS a major difference. Now at work you may drink the coffee from the break room, but it is not the great experience one is after, that is the convenience part of it all.

Listening is what I have always done and as JA said, the key to finding audio bliss, if one really cares about music being more than something in the background. Sadly, with interior car noise at about 60db on the road it may be that at least a 320kbps song might work, but for audiophiles a well done rebook cd is what we want.

I my recording studios I have headphones that I use for recording as I am listening for certain things, but I would never master a recording on them. I listen for pleasure on my AKG 701s and the higher end AT's. They sound totally different than my AKG K271s I use in the studio. I lost my Sony 7506's long ago.

I keep thinking that I should replace my old AR-58's which were the last remake of the vintage AR-3a, but after 3 woofer surround repairs and surround repairs for the midrange I have not found a reason to do so just by listening. Now the question is always is what is different from the 58's; better or just different? I have yet to find anything that I would consider "affordable" better than what I have. Diana Krall is right in my room right now and that is all could ever ask of a speaker to give me that "illusion" of real. The piano sound on this disc is what I hear when I play a real piano. What more could one want?

Now I find that what does help are files that are 2496 and 24192 that are well recorded and when I can hear those improvements on what my AR-58's can still do and can discern all of that over MP3's. I am content at nearly 72.

I have some great LPs, many new releases recorded and pressed well, but to play back vinyl well takes way more money than to play back great digital, and pressing quality can be all over the place as we all know. It is the stamping business after all. Many hands are in that soup.

I am happy for those who can own $10K vinyl playback gear and $20K speakers and know that are loving what they hear. I feel sorry for those who are not driven to spend even a little to hear more that the recording arts are supplying us now as music has never been presented better, but to many where fast food works it is sad that in 2019 they choose to have their music presented in that same way. I am OK with that...now.

I just did a recording for someone who wanted to send off a demo for a music competition who is very delusional about their ability. I do not charge people who come to my house wanting to improve their music making. One must hear what the audience hears to get better. It is not my job as a recording engineer to critique their performance, but to capture them the best I can. I make this comparison as I think this is much the same problem as some who think they sing and/or play adequately, who don't, and those who think what they are listening to is really, really good sounding. More often than not this is the case.

tonykaz's picture

People are Visual

Ears only contribute a small, nearly insignificant but important part of our Human data Collection. Hearing is almost a catalyst instead of the deciding component of a decision.

Zeiss in Germany claim that our Eyes do 98% of our Data Collection. Even one Eye will assure us a successful survival likeliness .

In our incredibly Noisy World where ambient Noise Floors can easily be well over 60 dba. people are resorting to noise cancelling technology for everyday use in public areas.

Our Auto Industry is now designing with NVH ( noise,vibration, harshness ) as determiners in design approvals. The quiet Tesla Cars seem the Best Choice for Audiophiles with Paul McGowan now using his as a Vlog Studio.

Advertisers are BLASTING their GODAM IMPORTANT MESSAGE well over our preset listening levels on nearly every Media transmission we care to access.

Dammit, the Noise is too dam noisy.

Now, people want less noise and less popular music like Michael Jackson or Beatles or any of the other Popular Music that gets played to DEATH.

MP3 means less noise for the majority of our population. ( I think )

We Audiophiles are the few that strive for Beauty from our Sounds, our Sounds are Art. We might be a dying breed. We certainly are a niche.

I arrange my personal home to have about 30 dba. ambient noise floors. , my listening area will be live end/ dead end design done beautifully with wood reflective surfaces up front and sound absorbing walls behind seating.

My Music will remain my life companion, a friendly comfort capable of Singing as well as my Childhood Mother the Operatic performer of breakfast Areas. I'll have my Starbuck One Pound bags of Coffee Collection displayed on a nice wood Shelve, in today's World, Audiophiles are the few that can afford the nicer things in Life. ( privately )

I'm hoping for a Change by supporting Bernie and Progressive concepts.

Tony in Michigan

jimtavegia's picture

The eyes can help you set up an audio system, help you position and find your seat, put discs in your CD player and help you align your TT cartridge, after that, closed or open eyes make no difference in audio. Too many have made mistakes buying audio on how things "look".

As for noise, trying to cancel is not the same as removal which is the proper engineering thing to do. If I have a 30 HZ or 40HZ noise, buying speakers that only go to 100HZ is not a solution. This would not be the same as actors who now want soft focus on their close-ups when the rest of us are using 4K and UHD TVs.

As for Bernie, we are a nation of Mathematic failures and following the footsteps of Greece and France is no solution with $22 trillion of debt and by 2020 or 2021 just the interest will be $1 trillion a year...what have we done to our children and grandchildren is evil and foolish. As a Math teacher I get it.

There has never been and never will be a "free lunch". People deserve what they earn and the meek and less fortunate are helped by me and my wife by the teaching of our faith as best we can ( I am still teaching at 72 and trying to make people employable), of which our government has no rudder (The USS Clueless). We are deceived by our greed. People do not deserve better government than they elect.

Bogolu Haranath's picture

You are correct, Jim T ..... Visual cues are different from auditory cues which are different from haptic cues and olfactory cues ........ One can read about 'sensory cues' in Wikipedia :-) .......

tonykaz's picture

I suppose that I agree with you, from your point of view.

But people are rather burned out by blaring Concerts, Blaring Car Audio playing RAP, Noise from Density of Populations, noise from dishwashers and especially noise from Corporate TV Networks. We live in a noisy world.

Pro Audio doesn't Cancel Noise but Consumer Bose does.

I can't figure out Greece, France or the EU and I have an Minor in Economics. I do realize that Greece's problems were exasperated by our de-regulated Banking Collapse in 2007. I can admire Germany and I admire an economic environment where citizens thrive. I'll spend the rest of my years promoting our youth's success. Go Bernie!!!

Most of the EU Countries have debt levels close to their Gross Asset Levels. Our USA Asset level is about 77 Trillion with our debt hovering around 22 Trillion with this current administration working to worsen our deteriorating position. ouch The Billionaires are grabbing more each day ( Hoarding it, there is no trickle down )

No free lunch ? tell that to the folks getting the Trillion Dollar Support ( Walmart, Billionaire Class, Wealthy Inheritors ). Walmart gets Corporate Welfare, Oil gets Federal Subsidies, Agriculture gets gigantic Federal Subsidies going all the way back to Earl Butts. Corporations Off-Shore to Mexico's $1.50 Labor, leaving their workforce in their little Towns hollowed out by Walmart selling high point Chinesium destined for Waste Management's Land Fill within 360 days.

I'm not preaching or predicting !

What is happening here is what happened in France back in 1800. When the top 10% own 90% of ALL the Assets, blood flows and it can't be stopped, even the Military joins the revolt.

I'm supporting Bernie, hoping to head-off a disaster. Is it too late?

Tony in Michigan

tnargs's picture

JA has been recycling that old Quad amp story for decades: I remember him using it in 2005 in a "Great Audiophile Debate" he had in opposition to the rationality of the late Arny Kreuger.

JA's epiphany story is really quite a humorous read. I also note that he has the arrogance to think that it represents persuasive evidence. Maybe it is an integral part of the nature that got him to where he is, but that sort of arrogance can really hold back the audiophile who is interested in making progress in his or her hobby. JA's subjectivist positions being a case in point of being held back.

Firstly, his understanding of the ongoing influence of sighted listening is lacking. Even if the blind test was experimentally valid (it may or may not have been), once he buys the Quad 405 and uses it at home, during that time, everyone around him in the audiophile community is still writing rave (sighted) reviews of the wonder and the thrill of using exotic gear. He keeps reading exotic gear reports and notices that the things they say (sighted) are not the same as the things he experiences at home: they are saying so much more than he is hearing. Furthermore, he continues to audition exotic gear (sighted) and has far more splendid experiences than what he hears at home with his 405. The reason being, the conscious mind cannot overrule the unconscious mind in creating these experiences. The fact that he was (maybe) telling himself consciously that they sound the same, does not overrule his unconscious mind continuing to create strongly different experiences, every time he indulges in sighted listening. This is the mounting dissatisfaction and unhappiness that he reported: not the result of sound waves emanating from the 405 at all.

Or not *necessarily* due to sound waves from the 405. Because this brings me to my second point. The Quad 405 was a dud. The 405-2 was a worthy classic, because it fixed numerous problems in the 405, but JA's experience in 1978 was years before the 405-2 was released. The original 405 used an op amp from the early days of op amps, the LM301, which did not measure well. The high input sensitivity could be overwhelmed by high quality preamps of the day with good output swing, especially, but not only, valve preamps. And the output stage was excessively current-limited, apparently to protect Quad ELS speakers, to only 3 or 4 amps. So it is entirely plausible to have a blind demo that doesn't stress the amp, hear 'nothing', then take it home and connect it to a more demanding preamp, or exotically demanding loudspeakers, or both, and suddenly you have an amp with issues.

In my opinion it is a pity that JA has such influence and status in the hifi hobby industry, with inappropriate logic and rationality dominating his preaching for decades.

Stereophile readers don't deserve a lot -- they are what they are -- but they deserve so much more than they have got from decades of flawed reporting.

Michael Fremer's picture

Your post reeks of arrogance and foolishness. You deserve nothing especially since what you've posted in response to JA's test taking makes no sense whatsoever. Whatever the fixes to the original Quad amp is not relevant to his original blind test result. I've taken many blind tests including at Harman's test facility and demonstrated the reliability of my listening, and both JA and I took part in a 5 amplifier blind test at an AES after I agreed to a challenge made by former Audio writer Dave Clark who claimed all amps that measure the same sound the same. I got all identifications correct and then though it wasn't part of the test, once we knew the identities of all five amps, I correctly identified ("blind") which was which. Nonetheless, though I scored 100% and JA scored 80%, because the population did no better than chance, we were both declared "outliers" and our results were tossed! What a joke. Interestingly, Clark included a VTL 300 tube amp in his test, an amp that measured and sounded very different than the others, which were solid state. Yet most of the AES crowd couldn't hear that one either! Why? Because under double blind conditions inexperienced test takers usually screw up and them did.

On the other hand it's now been proven that early humans survived tiger attacks by knowing precisely where they were coming from, through the use of "double blind" testing. Had we relied only on our senses we'd obviously now be extinct since "confirmational bias" would have produced the wrong location. Jesus you are a boring irrational fool.

AJ's picture

"I got all identifications correct and then though it wasn't part of the test, once we knew the identities of all five amps, I correctly identified ("blind") which was which. Nonetheless, though I scored 100% and JA scored 80%"

Thank you Michael for 100% confirming the viability of blind tests and forever dispelling the myth that they mysteriously "hide" the "real" differences that exist and only result in negatives/nulls. It's very funny to read otherwise ;-).
Yes, as you note, listener training and ability is important, but this too is well known by the scientifically literate http://harmanhowtolisten.blogspot.com/2011/01/welcome-to-how-to-listen.html
Harman for example, uses trained listeners as a subgroup when gathering large scale data. Basic hearing tests are key also.

Now quite frankly, if one just wants to "experience", audibly, visibly, tactilely, etc, etc a component, for overall satisfaction, then no "test" whatsoever is needed. Just mere subjective preference.

AJ
Soundfield Audio

Bogolu Haranath's picture

What about 'passing the smell test'? :-) .......

AJ's picture

Sure. We can't discount the smell of the Quad left JA dissatisfied vs the tube amp he ended up with. All part of the long term "experience".
We *can* however, dismiss it was the SOUND of the Quad, when JUST LISTENING and TRUST EARS. That had been clearly determined in the JL/TE aka "blind" test.
Obviously it was "other" factors during the long term experience, smell being but one possibility.

tnargs's picture

....for so perfectly exemplifying the Stereophile mindset problem. Not to mention a little name-calling and swearing, which is always a sure sign of an open and enquiring mind, and also gives us a little insight into what lies below the veneer of class that Stereophile self-promotes.

Storytelling is the art of myth-making. Rationality is the art of truth-making. You and I, we chose our respective arts long ago.

May I direct True Believers in the former art to some light porn: https://www.stereophile.com/taxonomy/term-a/162
Mr Fremer's name will inevitably come up.

Spend on and prosper.

Bogolu Haranath's picture

"Stereophile readers don't deserve a lot - they are what they are ...."

May be because, Stereophile readers practice 'Luciferianism'? :-) ........

dalethorn's picture

"Feel the Bern (burn?)"

Bogolu Haranath's picture

"Burn" ........ Ellie Goulding :-) ........

dalethorn's picture

Nice set of lungs, eh?

Bogolu Haranath's picture

May be nice everything :-) ........

Archimago's picture

Reading this article and your comment about the 2005 event with the late Arny Krueger simply reminds me how rigid and entrenched in the past these beliefs are despite the passage of time and progress in the technology.

A blind test of a Quad amp from 1978 (40+ years ago)?
JGH focusing on subjective assessment in 1962 (56+ years ago)?
JGH's comment from 1987 (31+ years ago)?
JA's own quote from 1980 (38+ years ago)?

I bet many of us reading this article had barely reached puberty when many of those words were written and can remember the state of the electronic products back in those days!

Subjectivity, being what it is, implies that nobody can deny that the person experienced what they experienced whether it's JA in 1978 or Fremer in his response. They are entitled to their opinions.

Those who try blind testing more recently will have had experiences just as valid as what JA or MF report. Some of us might even be a little humbled by the limits of our hearing rather than feel we were vindicated in the belief of owning "golden ears". Certainly in other areas of science, blind testing remains the gold standard to hopefully obtain unbiased knowledge from which a firm foundation can be built. I'm not sure how many readers accept that the observations of audiophile reviewers even in an esteemed publication like Stereophile feel that the conclusions reached are free of bias.

It seems that JGH himself regretted that the audiophile community did not embrace "basic honesty controls" since the 80's as per his interview here in 2007:

"... As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me, because I am associated by so many people with the mess my disciples made of spreading my gospel. For the record: I never, ever claimed that measurements don't matter. What I said (and very often, at that) was, they don't always tell the whole story. Not quite the same thing."

I personally think those were wise words.

Glotz's picture

Utter tripe. Wishful thinking.. mistrustful fools.

tnargs's picture

His soul was not for sale. He was not going to allow himself to become the object of derisive amusement among rational people. Respect.

He left a gaping hole. Which was quickly filled by those much more eager to peddle. A spiral began that continues today.

John Atkinson's picture
Archimago wrote:
It seems that JGH himself regretted that the audiophile community did not embrace "basic honesty controls" since the 80's as per his interview here in 2007:

"... As far as the real world is concerned, high-end audio lost its credibility during the 1980s, when it flatly refused to submit to the kind of basic honesty controls (double-blind testing, for example) that had legitimized every other serious scientific endeavor since Pascal. [This refusal] is a source of endless derisive amusement among rational people and of perpetual embarrassment for me . . ."

You can find the complete text of my interview with Gordon at www.stereophile.com/asweseeit/1107awsi/index.html.

I think it fair to point out that despite Gordon's strong language in my 2007 interview with him, he didn't perform blind testing to reach his review conclusions until I introduced blind testing for some of our loudspeaker reviews in the early 1990s.

He referred to those tests in this interview, when he said "Remember those loudspeaker shoot-outs we used to have during our annual writer gatherings in Santa Fe? The frequent occasions when various reviewers would repeatedly choose the same loudspeaker as their favorite (or least-favorite) model? That was all the proof needed that [blind] testing does work..."

Tom Norton and I worked very hard to develop the methodology for those blind tests and were complimented on it by none other than Stanley Lipshitz. However, when done optimally, that methodology was extraordinarily time-consuming and not really practical for a magazine that was published monthly. They were also not popular with readers, so as the blind speaker tests didn't produce results that were significantly different from our sighted listening, we abandoned them.

John Atkinson
Technical Editor, Stereophile

tnargs's picture

I agree that it would not have been practical, and I respect the decision not to go that way for a monthly magazine.

But the important question -- and I think the question that JGH did not much like the way it was answered in the actions of audiophile magazine reviews since the 90's -- is, if it's too hard to do right, "What now?".

Option 1: sell out. Pretend the placebo effect does not exist, and that perception effects like cognitive bias do not exist, or somehow do not apply to audio reviews. Report every impression gained from sighted listening as if it was all due to the sound waves. Pretend that if two or more people agree on an impression, that must prove it is in the sound waves. Behave arrogantly about beholding the truth, while burying it. Result: maximum magazine sales, maximum advertising revenue.

Option 2: cop out (with integrity). Go all modest and admit that you don't have a reviewing procedure that can isolate the audible effects of sound waves from other perceptual influences. Educate reviewers with rigour on the futility of trusting their ears when they know what the DUT is. Litter reviews with provisos and refrain from sharing personal listening impressions gained without validity or general applicability. Result: credibility undermined by the cop-out, readers unimpressed and leave, magazine closes.

Option 3: do it right. Properly controlled listening test regime. Magazine reduced to quarterly or less frequent, to allow time for testing. Reviewers educated and reporting as per Option 2 but with actual revelations to report. No cop-out. Strongly educate the audiophile readers on the realities on the influences of non-sonic factors on human perception, and how it applies to audio gear. Educate and strongly promote more about the basic longstanding body of knowledge about human responses to played-back music and human hearing limitations, and less about cherry-picking and over-hyping individual recent experiments that seem to conflict the known science (but in reality need a lot of validation). Result: magazine sales in between Option 1 and Option 2, advertising sales terrible due to so many reviews concluding no audible differences or no audible effect. Magazine moves to a semi-academic low-cost model despite high sales volumes to an appreciative audiophile audience who are finally learning the truth.

And although I understand the choice that was made, I am allowed to be sad about it, and to call it. Like JGH.

Regards

dalethorn's picture

So the choice seems to be between doing it right casual style, i.e. being patient and observing the artifacts, anomalies, and other sonic imperfections as you happen upon them in your listening, or doing it bureaucratic-style with an enforced regimen. I see Holt's great contribution coming from the days when he practiced the former. The latter has its place when done right, when the testers are sensitive to the reality that differences exist, even when the formal procedures aren't able to validate those differences - at that particular time anyway.

Ortofan's picture

... "when done optimally, that methodology was extraordinarily time-consuming and not really practical for a magazine that was published monthly. They were also not popular with readers ..."

Note that, nearly every month, Stereophile's sister publication HiFi-Choice includes the results of blind comparison tests of groups of various types of components. Is there something lacking in their methodology that allows them to regularly perform these tests? Would they keep conducting them and printing the results if such comparisons weren't popular with readers?

tnargs's picture

The first thing lacking, in their methodology, is a detailed description of their methodology. :)

I have seen their handwaving mention of "our blind panel testing", but reluctantly forced myself to ignore it because of the lack of transparency.

Like John said, it *does* take serious time, and gear-switching equipment, to set up controlled testing conditions and conduct tests that are legitimate and consistent with our best understanding of human listening abilities. You have to control for placebo effect, for human memory effects, for level (at what frequency or band?); have to provide enough runs to be statistically conclusive, etc etc.

For example, if their blind tests comprised 3 people in different chairs, and they played item A and let the music run until the listeners complete their scoring, then re-wired and switched to item B, C, ...F, then tabled the scores ..... that is a "blind listening panel" but is worse than useless.

Regards

dalethorn's picture

Argumentum ad Populum (or Numerum).

AJ's picture

"blind speaker tests didn't produce results that were significantly different from our sighted listening, we abandoned them."

Wow. That is in rather stark contrast to what science/Sean Olive et al have found.

http://seanolive.blogspot.com/2009/04/dishonesty-of-sighted-audio-product.html

How does one reconcile this?
Or the *significantly different* sighted vs blind result (yours) with say, the Quad 405?

Btw, as you know formal logic from what your professors taught, its Gordons *argument* that should be addressed, not "Gordon" per se.

cheers

AJ
Soundfield Audio

John Atkinson's picture
AJ wrote:
Wow. That is in rather stark contrast to what science/Sean Olive et al have found.

While I respect Sean Olive, I am not going to substitute his experience for my own. As I say in this essay, when I visit my reviewers, I hear what they have described. I regard their opinions as transportable.

Regarding blind testing, BTW, I performed my first in the spring of 1977 and have taken part in more than 100 since then. My opinions are based on that experience. You can find more on the subject at www.stereophile.com/content/simple-everything-appears-simple, where I wrote "Rigorous blind testing, if it is to produce valid results, thus becomes a lengthy and time-consuming affair using listeners who are experienced and comfortable with the test procedure. Otherwise, the results of the test become randomized, hence meaningless."

And I am not sure why critics like yourself continue to read Stereophile if you are so contemptuous of our reviews.

John Atkinson
Editor, Stereophile

AJ's picture

"While I respect Sean Olive, I am not going to substitute his experience for my own."
I wasn't asking for substitution. I'm positing his published, blind vs sighted experiment results conflict with your statement about your blind speaker tests coinciding with sighted preferences. Yet your blind vs sighted test results with the Quad seemed to square with his, i.e. markedly different. Seems inconsistent, to say the least.
None of which has anything to do with the widely accepted view by science since the days of Clever Hans, of the validity of controlled testing in every field of sensory science (and more).
I'll say again, if one wants to know how something sounds>ears, that is by definition, a blind listening test. If one wants to know what sort of effect the experience of a component will have on you long term, viewing, etc, then no blind test is needed whatsoever. There is zero reason to be conflicted about this and zero reason to come up with specious arguments against blind **listening** tests for **sound**. Two different type of experiences altogether.

"And I am not sure why critics like yourself continue to read Stereophile if you are so contemptuous of our reviews."

That is quite a mischaracterization. I have zero "contempt" for any review or reviewer. Their subjective experience, if one uses the literal meaning of the word, is subjective! Anecdotal and thus nothing contemptible possible.
The only thing I take issue with...and thus comment on, is articles like this that hold science (controlled tests) in contempt. Yes, I know that is a popular position these days, but I think it is totally unnecessary, given as I explained above, 2 different things/experieces.

AJ
Soundfield Audio

Jim Austin's picture

AJ, there are a couple of people on this thread who appear to be hard-core objectivists--that is, who deny the legitimacy of basic human experience. They appear to believe that the only route to knowledge is measurement, and they seem to assume that everything can be measured. Speaking as a scientist (former, but the training holds), I find such views quite naive (although I also don't believe these folks actually believe what they write; rather, I think they're just trying to harm the magazine). And from what you've written here, I don't think you share them.

I can assure you that John Atkinson is not anti-science. he is not on the side of (eg) the antivaxxers, and global-warming deniers. I've met other people who try to shove subjectivist audiophiles over to that side of those arguments, but it doesn't take. Most audiophiles believe strongly in science.

John's argument is that many scientistic folks underestimate the difficulty of this route to verifiable, scientific fact. They hold the naive view that everything can be measured. They fail to acknowledge that bias exists on the side of those who push their anti-subjectivist agenda, and that that bias works against the goal of authenticating subjective findings simply because they reject findings they don't agree with. (Did your listening test prove you could tell the difference between those cables? Then obviously the test was flawed!)

It seems to me that you and JA are very close to believing the same thing. You acknowledge--I think, in this latest comment--that there's validity to subjective testing. What you object to is (your perception of) JA's rejection of science--but he's not rejecting science; he's saying it's hard, and that its advocates are often at least as biased--as unwilling to be contradicted--as the subjectivists.

They also fail to understand--and you seem to understand this--that the enjoyment of music is a deeply human, inherently emotional thing, which, at a minimum, legitimizes other, non-scientific perspectives. Science has no hegemony in this realm. They don't seem to understand that in rejecting subjective testing they're simultaneously rejecting much that makes life worth living. Children don't use the scientific method to learn to walk, and talk, and exist in a complicated world. Grownups don't do p-tests in choosing a life partner. You don't need to understand blackbody radiation to enjoy a sunset.

Oh, and I should add this: Is there uncertainty in this kind of knowledge? Of course there is. But to reject it is to reject too much.

Be Well,

Jim Austin, Editor
Stereophile

AJ's picture

"It seems to me that you and JA are very close to believing the same thing. You acknowledge--I think, in this latest comment--that there's validity to subjective testing. What you object to is (your perception of) JA's rejection of science--but he's not rejecting science; he's saying it's hard, and that its advocates are often at least as biased--as unwilling to be contradicted--as the subjectivists."

Yes, I'm saying ultimately, if a $100k cable, or a tube amp of high repute makes you happiest, after a years experience, then by all means enjoy! If a reviewer swoons over a product, great, that's their experience. Nothing "contemptible" at all, as John has accused me of.
What I *do* hold in contempt, is that there must be this dichotomy where to accept any of the above, one must dismiss the validity of 100 years of sensory science. That somehow a blind listening test sound>ears, is invalid, because a 1 year viewing/touching/listening/reading stuff/etc experience had a different result.
That, I find anti-scientific and absurd. They are 2 totally different things. Nor would I be humiliated to find that I actually didn't prefer the taste of my favorite wine in a blind test.
Why? Because it wouldn't matter to me. I don't normally drink wine blind! So I fully grasp that it is the entire experience, my beliefs, **including the taste**, that makes me happiest.
And I most certainly wouldn't dismiss the results as invalid for taste. That, would be anti-scientific.
These absurd, false dichotomies in high end audio are totally unnecessary, given that controlled listening tests and casual for-enjoyment experiences, which include listening, are in any way incompatible.

cheers,

AJ
Soundfield Audio

p.s. I have tube, Class A, Class D amps, expensive cables, etc, etc. in use regularly at home.
I make no silly excuses as to why I like/dislike them....subjectively.
It's my preference, 'tis all ;-)

RH's picture

Jim,

When you characterize "hard-core objectivists" as "denying the legitimacy of basic human experience" that comes off as something of a self-serving characterization; a straw man.

When medical trials are undertaken under double bind, placebo-controlled conditions, is that a case of "denying the legitimacy of the basic human experience?" Or is it not simply being responsible, in light of what we've learned about how human bias becomes a confounding factor in weeding out cause/effect?

Surely it's the latter.

Objectivists tend to point out that there is no reason to think high end audio would be magically excepted from the problems of human bias - and in fact we have every reason, including scientific reasons, to think it to be a significant confounding factor.
So it makes sense to control for it if you are really trying to understand a phenomenon...just as one would do in a drug trial.

No, it's not easy. But then how are even *looser* and less controlled methods the better choice??

(BTW, I'm not talking about every audiophile turning his purchase in to a science experiment using double blind testing. That's obviously impractical for lots...though not all...gear. And people can buy whatever they want for whatever reason they want. I'm just talking about *when* we really want to understand a phenomenon, what does it look like when we are being scrupulous and careful? It looks like science.

The problem with pointing to scientific bona fides, or even just a "respect or love" for science, is that it doesn't at all guarantee someone isn't falling in to error. After all, as Feynman cautioned us: The first principle is that you must not fool yourself—and you are the easiest person to fool.

That's why of course the work of great scientists requires double-checking by other scientific parties like any other scientist.

And the evidence that people who love science, or who are great scientists, can lower their scientific scruples to let an agreeable or bias-confirming belief hop over is all around us. (Note how there are scientists who produce great science following rigorous empirical scruples, who go home, open an ancient book and accept ancient claims of miracles. People are great at making exceptions for beliefs or experiences they like).

So along those lines, though JA may be (and often is) empirically scrupulous in various areas of the hobby, his rejection of blind tests seems to read essentially:

We've undergone blind tests to see if our sighted impressions are reliable. The more strictly controlled blind testing yeilded results that suggest our sighted impressions were not reliable. Therefore, we reject the blind tests, presuming our sighted tests are more reliable."

In other words, circular question-begging, it seems to me. And, more worrying, of a type that one finds virtually every crack-pot pseudo-scientist, psychic researcher, New Age therapist, etc, making. "The science can't confirm our impressions, so we reject the science."

I don't put JA in "crack-pot" territory - I have huge respect for him and what he's provided us over the years. But I think it should be worrying once you have appealed to the type of reasons that are also used by people to support absurd claims.

I think the "objectivists" wouldn't care much if the subjective character of reviews were justified on any number of pragmatic grounds. The problem comes when the pushbback involves making claims about the scientific approach, and it's application to audio, that are dubious. Note that you can take all the problems JA purportedly identified with blind testing - confounding variables of a listener's mood etc. - and it's reasonable to be left asking "so...the BETTER approach is to just STOP TRYING to control for confounding variables?" Because then it seems you have all the problems JA sighted PLUS an even looser approach to controlling for variables.

I very much enjoy Stereophile, and personally I enjoy the reviews and have some of the subjective reviews to be useful, and "accurate" to my own personal perception of the production in question. So I'm not one who rejects subjective reviews out of hand. But...I can't ignore their liabilities either.

Jim Austin's picture

I didn't set it up as a straw man, but as a legitimate point. Many of the people on the objectivist side of these debates either deny the legitimacy of, or fail even to acknowledge, that scientific knowledge isn't the only kind of knowledge--that for all it's virtues, science is actually pretty rare when it comes to informing the human experience. If you leave out the scientific part, you're not left with antivax, shark venom, or climate-change denialism. You're left with most of what makes life rich (plus those things).

Here's what I think--and I'm the Editor now. I welcome science. I love science. Over time I've come to understand and embrace the subjective approach as well--to realize that there's more than one legitimate approach to analyzing emotional experience facilitated by technical means. This is important--and Stereophile is and will remain a subjectivist audio publication. But I will never lose my appreciation for science.

I find this discussion encouraging, because it suggests that part of the problem is people misunderstanding each other. From my perspective, Stereophile's critics are mainly criticizing the magazine's subjective approach; this business about us rejecting science is new, or at least uncommon. My problem with them is precisely that they seem to reject the subjective approach--that they view the magazine's subjectivism as intrinsically anti-scientific. If those critics were to accept the legitimacy of subjective reviewing, we'd have little to disagree about.

Concerns about the applicability of science in this realm are legitimate however--and it's not anti-scientific to say so. People love to use clinical trials as an example of this ideal. This is something I know about from my science-writing days. Traditional clinical trials average over the responses of many individuals. Even in medicine this is problematic, since not everyone responds to drugs in the same way. A drug that works for a small number of people (because of unique genetics, unique gastrointestinal flora, or whatever) won't get approved. If you take the same approach in audio, the people who can hear get lumped together with the people who can't hear. There's an essential difference here: The listeners aren't the subjects; you're not trying to learn something about them. You're trying to learn about the phenomenon--is it audible or not? All it takes is one person who can hear it to prove that it's audible. Such methods don't acknowledge that. (Newer methods of clinical trials acknowledge differences among participants and dive deeper into the data for insights.)

As I'm sure he would acknowledge, much of the work done by Sean Olive involves relatively large differences, like frequency-response differences among different headphones or loudspeakers. Trying to scientifically establish whether similar amplifiers (with nearly identical frequency response) can be differentiated is a different order of problem from a statistical perspective. It presents different analytical challenges. And--again--on those rare occasions when real evidence emerges, it's often dismissed for bad reasons. To call certain kinds of science difficult is not to dismiss science.

I believe in science--and I know how difficult it can be to do it well. The subjective approach to judging the effectiveness of a component in delivering an emotional experience is legitimate. I will continue Stereophile's policy of encourage its writers to express their own opinions, but if Stereophile has an official position, that's it.

I'm out.

Jim Austin, Editor
Stereophile

Bogolu Haranath's picture

"Absence of (scientific) evidence is not evidence of absence" :-) ........

ok's picture

we should all leave the “science” word kindly out of this debate. There’s nothing rigorously scientific in double, triple or whatever degree of blind audio testing other than the kind of “3 out of 4 people definitely preferred the voice of Justin Bieber over the one of Bob Dylan” or "It all sounds the same to me, don't they?" results which 1) leave the arbitrary selection/evaluation of the test subject at the mercy of the tester's statistical wizardry –I'm pointing at you Harman– and 2) tell me absolutely nothing about my personal sensibilities and preferences which may or may not fall into the majority’s bell curve sweet spot; it's like inferring my own IQ based solely on other people's test scores. There is nothing objective whatsoever in the statistical quantification of human taste nor is there any value in blind testing enchantment other than deconstructing the credibility of evil reviewers –which is not a bad thing by the way, provided that is accompanied by a similarly thorough deconstuction of pseudo-scientific audio psychometrics. As far as "science" is concerned, for all that's worth I'm perfectly happy with some basic hardware measurements; mine ears can do the rest of the job, thank you very much. Subjective reviews are an essential part of our hobby and they should keep it this way –subjective that is– no less no more. Even if all these reviews were nothing but corrupted payoffs as rumored by some, I frankly don't see what in the world could prevent any form of blind testing from following suit save for being performed at the presence of the police.

dalethorn's picture

"The only thing I take issue with...and thus comment on, is articles like this that hold science (controlled tests) in contempt."

I don't read a contempt of Science or Controlled Tests. What I do read is a contempt of blind faith in "science", which is not the same as Scientific Method. Also a contempt for controlled tests which deny that which is heard, simply because a controlled test doesn't hear the same thing due to irreconcilable listening methodologies.

Bogolu Haranath's picture

"Somebody's Watching Me" .......... Rockwell :-) .........

dalethorn's picture

"I'll be watching you" - the Police.

Bogolu Haranath's picture

'Watchmen' ......... Movie soundtrack :-) .........

AJ's picture

"All I could conclude was that the blind-test protocol itself had become what scientists call an "interfering variable"—that the conditions of the test were too far removed from how we listen to music through our systems to give meaningful results."
Correct. The blind/controlled JUST LISTENING and TRUST YOUR EARS tests indeed prevent self deception from affecting the meaningful SOUND results.
If one want's to find what psychogenic "meaningful results" occur when one stares at/touches a Quad 405 or whatever, while reading poor subjective reviews for a year, then no blind test is needed.
The blind listening test is purely to determine the sound of the DUT in a trust your ears scenario.
The long term viewing/touching/knowing/etc/etc "experience" will reveal the full psychological impact of the DUT, without any need to "trust ears" and "Just listen".
Two different things entirely. Words (Sound, Listening, etc) have meanings, unfortunately.

cheers,

AJ
Soundfield Audio

p.s. I thought you weren't "The Editor" any more...

Jim Austin's picture
>>p.s. I thought you weren't "The Editor" any more...<< This is a repost from the May issue of Stereophile, which was laid out in March. My first official issue as editor is the July issue, which will be available in mid-June. Jim Austin, Editor Stereophile
AJ's picture

Thanks Jim, I swore I had just read the other JA was now "The Editor".
Are those initials a prerequisite? I'm pretty close....

Jim Austin's picture
>> I'm pretty close....<< You're inverted--opposite--backward. It would never work. ;-)
Glotz's picture

That PS sounds like a 12 year old in a old man's body.

AJ's picture

What power supply????

ok's picture

..one thing consistently overlooked concerning the much maligned “placebo effect” is that in so-called "failed" blind medical experiments a statistically significant percentage of patients gets literally healed merely out of thin air. No, it’s not their illusion –it’s the actual medicine that fails to cure them.

tnargs's picture

So the analogy being, if it's the actual medicine that fails to cure, then by analogy the actual sound waves fail to satisfy, but only the magical effect of the lies that we have been told about the playback gear, i.e. the placebo deception, that causes us to have any pleasure.

You are right that there is a placebo effect, but this only dictates the necessity for controlled medical tests (if you want to know if the medicine works), and controlled playback listening tests (if you want to know if the sound waves themselves are preferred).

So yes, the placebo effect is real, and just as the placebo medicine effect actually sometimes cures without any actual medicine, then also the placebo playback-gear effect actually sometimes brings pleasure even when the gear is doing nothing audible to the sound waves. BUT... there is also medicine that does cure, and there is also audio gear that does produce preferred sound waves, and in both cases the only way to find these items is by controlled testing regimes.

The problem with the Stereophile model of reportage is that they don't believe this basic science. They don't believe the placebo effect exists, they certainly don't believe that it applies to them personally, and they joyously expand on every anecdotal story (see my earlier comment about myth-making) of someone (especially themselves) 'passing' a blind test, and they give zero (or less) coverage to the basic science of the placebo effect and how it is dominating the findings about equipment that they are reviewing as if it were all caused by the sound waves.

Look at Stereophile founder JGH's latter interviews if you want to know his perspective on this. The disservice is huge, and he felt it in his bones, and we have the likes of JA, Fremer, Dudley, Reichert, and a string of others to thank for perpetuating it. And if you want to see how much they like being "called out" for doing this, check Fremer's response to my first comment on this article.

Spend on.

ok's picture

in the aforementioned experiments is also the paradoxical fact that the placebo-driven patients actually base their healing illusion on the undisputed power of scientific authority..

rschryer's picture

...for doing this, check Fremer's response..."

Stereophile isn't the Borg, bro. Each of its writers has his own temperament and personal likes and dislikes, but all take their reviewing jobs seriously.

Just as you, apparently, take your finger-pointing, party-crashing, smug-dude job seriously.

No one is trying to assimilate you, tnargs, just as no one invited you here. If Stereophile post-Holt is a joke to you, you are free to leave our house and spread your brand of empty joy elsewhere.

Move on.

tonykaz's picture

I have a "feeling" that you have something profound to offer and present, I'd like to see your reasoned contributions. Right here there is nearly unlimited Space for you to detail your thoughtful methodology in all Audio related matters.

Please keep in mind that when two people agree on something, only ONE is doing the thinking. I would like to read your points of view and ask your acceptance of any thoughtful & Honest comments I or any other person might have.

This Place is a Campfire setting which you are welcome to have a seat, listen in, make your comments and even challange things.

The entire Audiophile World is reading your posts, no-one here is your adversary, there is plenty of elbow-room for everyone.

Personally, I'm hoping that you are an MP3 loving, non 33.3 person so that I will have at least one other person that is as distant from established Audiophile Norms as I am.

So, relax, state you reasoned opinion ( we all have them and don't agree on most things ) , have a good time challenge AB testing, Double Blind testing , validity of Speaker Cable Trusses, Class D Amplification and Pleeeeessseeee help me challenge/confront the invasion of chinesium in our fragile little Audiophile Industry.

Tony in Michigan

ps. help us explore the entire World of Home Audio

dalethorn's picture

Most medicine today is black magic, supported by patient statistics and willing investors.

tonykaz's picture

My Retail customers heavily weighted their purchases on Reviews from TAS!

When TAS blessed the Monster Alpha 1 phono Cartridge, I sold 20 within a week of that Issue hitting the Mailboxes. ( Koetsu was far better )

So, a good percentage of Audiophiles believe the Written Words they SEE from Reviewers they "trust" more than they "trust" their own listening, for good reason ! My Customers ( and me at the time ) didn't have scads of Gear experience, we had Kenwood 30 Watt Reciever experience.

Plus, We all love that "Higher Authority" kind of blessing HP could give a product, don't we?

And then comes "Group Think" like what happened with Monster's Reference interconnects and AB Testing of Monster's Speaker Cable ( I had the Sales Counter Device that Monster Created to test Speaker cables, everyone believed in and purchased Monster Cable )

Stereophile's Recommended Ratings

Don't we aspire to own only the Gear that achieves A or A+ Reviewer acceptance? I know better but I've owned thousands of carefully selected pieces of audio gear. I love that lower performing Linn LP12, I love the Schiit Headphone Amps ( Asgard 2 & Valhalla 2 which are also high performance Preamps ), I admire a wide range of gear that isn't even Rated by Stereophile. Hmm, but then again, I have "earned confidence" yet not enough to evaluate DACs which all seem pretty much wonderful despite what Reviewers end up revealing. I love the vintage Sennheiser HD 580/600/650 headphone that Steve G. originally recommended back in 2011, he was right then and is still right. Reviewers today are recommending multi-thousand dollar headphones.

We are beginning to be offered "Audiophile" Grade Wireless Loudspeakers, I'm excited to be reading HR's considered & informed reporting. Imagine a Wireless World, it might be too much to hope for. I have my fingers crossed.

Tony in Michigan

Ortofan's picture

... the lights would go out at all of the brick and mortar audio retailers who depend upon the sales of high profit margin interconnect cables just to stay in business.

Bogolu Haranath's picture

Also, expensive power cords, power conditioners, power generators and re-generators, power outlets, super capacitors, rechargeable internal batteries ........ not to mention cable elevators ...... sticky things that stick on speakers ,,,,,,,, will stay in business and flourish ....... I might have forgotten few things :-) .......

tonykaz's picture

like the St.Anthony Holy Water Soap to properly wash hands before touching Audio Gear. Prevents & Neutralizes contaminants from migrating thru openings in the Electronics.

I suggest you to be one of my Early Ground Floor Investors in this important accessory.

Remember , all the profit is in P&A ( parts & accessories )

Tony in Michigan

ps. I found Magellan's Fountain of Youth and am looking for investors to help me get bottling and distributing. I encourage you to choose your poison!

Bogolu Haranath's picture

There is no cure for 'audiophilia nervosa' :-) ..........

tonykaz's picture

... good advice from a Radiologist. ouch

Tony in Michigan

tonykaz's picture

You are painfully correct in this assessment.

My last months of Brick & Mortar Retailing were financed from our Monster Cable Sales.

Pro Audio embraces Wire and Sells 500 foot spools of the stuff and probably always will.

Audiophiles will probably discover the Pro Audio Outlets as the last viable Retail Audio Vendors.

Audiophile is a Fragile Industry.

Tony in Michigan

dalethorn's picture

"We all love that "Higher Authority" kind of blessing HP could give a product, don't we?"

Who's more reliable in the real world, HP, or some lying politician who says "feel the bern"?

tonykaz's picture

ouch,

Tony in transit

dalethorn's picture

I may not trust 'em but I love 'em anyway.

tonykaz's picture

Probably the nicest picture in any audio Magazine anywhere.

Superb click-bait

supports the concept of buying with eyes.

Nice work from the Artist creating this !

Tony in Michigan

Bogolu Haranath's picture

"Eyes Without a Face" ........ Billy Idol :-) .........

Bogolu Haranath's picture

If we look at the KEF LS-50 Nocturnes, they look remarkably similar to those eyes :-) .........

Bogolu Haranath's picture

...... also, the blue LSX and the album cover of the May 2019 recording of the month and the blue meters of the Mac amp ...... well done guys :-) ......

dalethorn's picture

I never thought I'd write about this, but given the above, what's to lose? I'm a headphone guy, or been so since circa 2009 when I got the Sennheiser HD800. As I tried a lot of different headphones, their sonic differences generally outweighed everything else, assuming good test recordings. Finally I found a decent parametric equalizer, and tried bringing some of these headsets a little closer to each other in sound, to see if there was something underlying the mere differences in frequency response.

Skipping ahead, and without my getting involved in any complex listening projects, what I most noticed between the different headphones and the other gear (DACs, amps) was the sense of realism - partly I suppose the soundstage, or the accuracy of the background ambiance. But there was something else - the upper harmonics and sense of "air", which I assume needs a decent and even response up to at least 14 khz. I say 14 khz because even the premium headphones are wildly different once you get above 10 khz.

Anyway, whether speakers in a real room or headphones on a variety of different heads, the task of getting the extended harmonics right and in proper balance isn't a slam dunk unless the audiophile has control of all the important factors. Power filtering can make a huge difference, but with headphones and battery-operated amps, I can get around that issue. Shielding can make a difference, and that costs money when you look at all the places where it applies.

For actual testing, I've found that I can play a musical segment of a few seconds to half a minute on test rig 'A', then repeat the same on 'B', and after a few rounds usually hear the differences. If not, I'll take a break and resume later in the day when my attention is better. There are very few things I've ever owned where switching amps didn't make an obvious difference, and in some cases even connecting cables and especially headphone cables. If you're an experienced audiophile who's curious and inclined to exploring these differences, you already know that the differences exist, so it's just a matter of having the patience and commitment to find them.

Jim Austin's picture

There are plenty of audio-related forums on the Internet where you can post attacks on the character of Stereophlle's writers, editors, and others you disagree with. I will not allow it here. I have deleted your comment.

I do not object to criticisms of Stereophile's editorial philosophy. Feel free to make your case. But if you are going to make them our site, you're going to need to find a way to do it without attacking people's character and questioning their integrity. Simple as that.

Jim Austin, Editor
Stereophile

tnargs's picture

Two posts out of two, I have to encourage you to read with comprehension. I gave two scenarios: one where your writers and editors lack integrity, the other where you all honestly believe what you write. And I clearly wrote that I believe the latter. Didn't I?

Simple as that.

So, it behooves you to undelete my comment. Perhaps even to admit your mistake.

cheers

Jim Austin's picture
Have you stopped beating your wife? Your arrogance is amusing, but it is not effective here.
tnargs's picture

Did I or didn't I write that I believe you guys are genuinely sincere?

The fact that there are ramifications flowing on from that sincerity, which you are paraphrasing as "have you stopped beating your wife", is an editorial philosophy discussion, which you purportedly welcome, and not an attack on anyone's integrity.

I repeat: if I say that you are genuinely sincere in holding beliefs that are generally held to be wrong outside of the bubble world of audiophilia, that is an attack on philosophy, not integrity.

So undelete it, please. As a matter of integrity. My post was never anything other than a criticism of Stereophile editorial philosophy. You have committed yourself to welcoming that.

Jim Austin's picture

I'll make three points clear as crystal:

1. You are posing as a judge. You are not a judge. You don't get to decide.
2. The alternative you offered to a lack of ethics and character was still insulting. Instead of thinking of various ways that we could be wrong, you should be thinking of ways that you could be wrong.
3. You're getting annoying and taking up too much of my time. Henceforth I will ignore your comments unless they cross a line, and then I will delete them.

Jim Austin, Editor
Stereophile

tnargs's picture

Yes, that's probably a good thing. You seem to be more focused on attacking my behaviours than addressing any issues with the Stereophile approach, so no point in continuing. Since you gave me the last word, I'll also finish with 3 points clear as crystal:

1. Every time I ask you to confirm that I never said you guys lack integrity, and that I did say that I believe you are all sincere, you avoid directly answering and try a new dig at my character, so I think any unbiased reader will know by now that my deleted post never said what you claimed.

2. If you want to see some actual insults and attacks on integrity, read Fremer's response to my first comment, https://www.stereophile.com/comment/583330#comment-583330 . Now imagine how you would have treated me if I had said that about him. Now consider whether both you and Fremer (with his swearing and name-calling) are chronically over-reacting to fair criticism, simply because it is hard to take.

3. I come from a culture that is known for a straight-talking and no-nonsense style. So, mea culpa for not being ingratiating. Yet I do not resort to swearing and name-calling. If you want to see a better standard of discourse, perhaps a little staff training is a better place to start, instead of attacking and censuring your more-critical correspondents on any pretext.

cheers

ChrisS's picture

...in the entire audio industry does blind testing the way tnargs thinks Stereophile should.

X