How Does the Music Make You Feel?

Stereophile's first change in editorial leadership in 33 years calls for a restatement of the magazine's core principles.

Stereophile was founded in 1962 by J. Gordon Holt, on the premise that the best way to review an audio component is to listen to it. Following Holt as editor, John Atkinson turned that premise into a viable concern—a real magazine—and, in 1989, added a regular suite of measurements to Stereophile's otherwise subjective mix.

With his commitment to listening first, JGH created a new genre of audio publications; many others followed in its wake, of which only a handful survive. For Holt, it was a question of expediency—measurements weren't telling the whole story. JGH was not against science. On the contrary, he embraced the pursuit of quantifiables, if inconsistently, and regretted it when the subjectivist-critical world he'd created lost its bearings. Perhaps I flatter myself, but in this respect JGH seems like a kindred spirit.

I've spent most of my professional life doing science or writing about it—but this task is different. Music isn't science, and the experience of music—its emotional impact—is Stereophile's chief concern. For the moment, forget about the bits, bytes, and electrons, the clever firmware encoded on field-programable gate arrays and assess how the music makes you feel. Then switch out a component—just one, since we do have a methodology—and do it again.

There's no doubt that we owe a debt to science-based designers of fine audio equipment. But it's our right as music lovers to judge how successful a component is in conveying music's emotion. Were you deeply moved or left cold? Fifty-seven years since JGH's audacious experiment, I suppose we could, if we wished, apply science to that question, perhaps by hooking listeners up to an MRI machine as they listen to the same music with different DACs or preamps. That would be interesting, but it would be too expensive to do routinely, and it would probably result in boring copy.

Stereophile's reviews are what you might call narrative listening journals, documents of sonic/musical experience composed by skilled listener-writers. As a college professor taught me years ago when I was still studying theater: Watch, respond, and figure out what it was that made you respond that way. Substitute "watch" with "listen"—that's what Stereophile does.

The argument objectivists make against this approach is that it's soft and uncertain—too many sources of bias: How can you be objective? But that's just it: we can't. The same humanity that gives us standing in assessing how well emotion is conveyed renders us imperfect judges of said conveyance and the reasons for it. Ours is a human assessment, not a scientific one. But is this not also true for many other things—living, eating, loving?

Learning to assess audio components is a bit like learning how to live: Pay attention to how you move through the world. Assess your own responses. Develop your sensibilities. Accept uncertainty as a fact of life. When was the last time you attempted to measure how much you love your children?

Let us not, however, give the floor over entirely to the artsy music-appreciator crowd. Despite my just-stated humanist perspective, I have a little bit of scientist pride left.

Hence, measurements.

Measuring error may be low, but there is much uncertainty in measurements' relevance to the conveyance of emotion. Subjective assessment is uncertain, but its connection to what we wish to assess—our emotional reaction to music—could not be more direct.

And speaking of uncertainty: We're not against statistically valid listening tests. We're just more aware than most of the challenges of doing them well and their tendency, as typically performed, toward missing effects that are subtle but real. Plus, as the cliché goes, absence of evidence is not evidence of absence.

Stereophile, then, is a subjectivist journal that measures. As Herb Reichert would likely point out, listening is our yin (bringing spirit) and measurements our yang (offering form).

We are fortunate to have on our team some of the best ears in the business, allied with the best pens—and one of the best and most experienced measurers of fine audio equipment. This potent combination is why Stereophile is the most popular high-end audio magazine in the world.

This column is about values, so let me not forget to mention some other important ones. Stereophile practices an old-fashioned, strict separation between its editorial and business sides. You can't buy a review in Stereophile. No advertising contract or other financial transaction can ensure a Stereophile review. The best way to get a review is to make an interesting product that promises excellent sound. Good value matters, too, and so does having a track record of excellent customer support.

Editorial independence gives Stereophile a credibility that keeps readers coming back. It's not just readers, though: Our integrity is one reason a good review in Stereophile means as much as it does, and why ads in Stereophile still have impact.

A final point: Stereophile writers hold a wide range of opinions. I don't tell them what to think or write. With thanks to my colleague Jon Iverson for providing the metaphor: Stereophile is a clubhouse where people can come together over a shared love of music—but with widely disparate views on pretty much everything else: tubes or transistors, classical or rock'n'roll, digital or analog. We gather to respectfully but energetically discuss and engage—then toast our shared love of music with a favored beverage, beer or scotch or protein shake.

Welcome to the club. Take off your shoes. Stay a while.—Jim Austin

COMMENTS
Anton's picture

As an audiophile, all I need to do is sit back, relax, and be a subjectivist.

I don't need to measure anything, I don't have to design anything, all I have to do is consume the music, dance, and be happy!

Surely, I would not want a pure subjectivist to design or build my gear. I can't imagine a pure subjectivist-designed piece of kit: "This capacitor is pretty, and I don't like the connotation of the word 'resistor' at all. Impedance sounds like it might hold back music, let's got with zero impedance..."

So, thank God for objective measures in construction and design!

All I gotta do is either dig it, or not! Yay! Thank you!

Put me down as someone who realizes it takes "both" flavors to make the hobby work. As Laozi, said, "It takes both yin and yang to make the Tao go round!"

I guess, then, people could start to argue about how rigorously we apply our subjectiveness in an attempt to be subjectively objective, and that's fun and all until someone tries to make a claim that 'has to' apply to all listeners. Then, it becomes a religious argument and saps the joy.

But, then, I consider limoncello to be a "system enhancing solution," so take this all with a drop of yin, or yang, or whatever it is floats your boat.

ishis's picture

Thank God? What mindless silliness!
Thank Stereophile for doing measurements, not some imaginary, irrelevant deity. (that is a very annoying expression that should be stricken from the language of intelligent people)

Anton's picture

Suck it?

;-P

avanti1960's picture

since the beginning of human existence been able to determine the exact mechanism of the creation of the universe?
Have they been able to completely map and identify the composition of the building blocks of physical matter including subatomic particles, energies and their relationships?
Since the answer to both questions is no- how is it that the collection of the the most intelligent people in the earth's history haven't been able to understand them- yet those that do not believe in a creator believe they came from nothingness through random chance and accident?
The history of physical science is stumped but it happened by accident....?

Anton's picture

There are some people who can simply say "Thank God" as a figure of speech and not imply it as an endorsement of the supernatural.

It is a phrase used as an interjection to express relief, thankfulness, etc.

If I said, "holy cow!" would people think I was endorsing Hinduism, or that I venerate cows as spiritual epitomes?

This makes me wonder, are there people out there who think that the Christian savior's full name is "Jesus H Christ?"

Do they think we believe in the Devil if someone says, "Bloody hell?"

Come on, people.

Bogolu Haranath's picture

'Thank God' there is Stereophile ....... 'Holy cow' there are so many objectivists ....... What the 'bloody Hell'? :-) ..........

RH's picture

You can appeal to the supernatural to "explain" anything.

That's a "bug," not a "feature."

RoryB's picture

Did someone with a Christian fish on their car cut you off in traffic? Or are you always just this unpleasant and quick to take offense? You don't have any special information about God's existence (or not) that the rest of mankind doesn't have, and still to this day it is up to each person to decide what they think about this, not up to you.

Jim Austin's picture
Let's steer this conversation way from religion, shall we? Also: politics. Jim Austin, Editor Stereophile
Bogolu Haranath's picture

Music is all about feeling :-) ............

Robin Landseadel's picture

What I like around here is that it's subjective + measurements. I'd like to see more of that everywhere. The issue, for me, is finding out what specific measurements correlate to good sound. "If it measures good and sounds bad, -- it is bad. If it sounds good and measures bad, -- you've measured the wrong thing." So this process, which Stereophile has been better at for longer than anyone else, is in part attempting to find out how measurements correlate to sound quality, which is all by its lonesome a worthy endevor.

Of course, what is considered "Good Sound" is, ultimately, subjective in the very narrow sense of really belonging to only one listener. Lots of folks are looking for a reasonable recreation of "live sound". Different for a headbanger than a Mozartian.

Once I heard the colorations of microphones independent of a mix + master + gear + room , I became aware that you can't really "get there from here". When monitoring a performance for a recording, I'd take the headphones off, walk into the room where the musicians are performing, and enter a different world. And it didn't matter if the microphones were mine or somebody else's, same thing. The best recording engineer I worked with used monitoring gear that would be comfortable for long sessions [if headphones] and not burn your ears off. I remember going to Skywalker, WATTs in place, auditioning a live recording that was recorded/played back via a Panasonic 3700. Took 15 seconds before the engineer/producer decided to replace them with a cozy pair of British Mini-Monitors.

Surviving the session is more important than maximum fidelity.

I don't expect to hear "live sound" from a melange of microphones and black boxes with blinking leds. What I hear now [if I'm listening 'critically'] are a lot of the artifacts of a given recording/playback chain. That's what attempting to make a living as a recording engineer has done to me. I am happier to hear a reasonable rendition of a microphone feed than a nearly plausible representation of "live music". If I want "live music" I play some or get out of the house and hear some. But a lot of attempts at a simulacrum of the real thing make me aware of the attempt, which breaks the spell.

When I'm playing my Martin I don't have to wonder if the sound is real. When I'm listening to music through a Marantz 8b, I hear how the resolution is nearly perfect in the mid-range but loses its grip at frequency extremes. I always hear the deliberate hardness of the sonic presentation of Klipsch speakers [nice dynamics, though], always hear the audible compression of classic acoustic suspension speakers [nice mids and bass, though]. Whenever I listen to an LP these days, I notice how the "gas"---resolution, bass, treble, dynamics---runs out before a side is over. "It's Always Something", like Roseanne Roseannadanna always sez.

On the other hand, the stuff that used to bother me [a lot] about "digital sound" feels like it's fading out. Once 24/192 became the de-facto standard for DACs, things improved "tout suite".

Right now, I'm looking for minimum irritation in music playback. I've got hours & hours of time at work where I'm listening to my personal stereo. This is one aspect of audio that has become considerably less irritating over time. The notion of having access to nearly every record in the world via something you can stick in your pocket or handbag is some kind of a dream. The notion that one of these little devices will be, in the near future, the best sounding of any source components is my dream right now. I can think of many aspects of digital audio that would be best realized in a battery-powered device working off of micro sd memory chips, or whatever the "next thing" in storage is that's gonna be faster, denser and physically closer to the DAC. I'm looking forward to that.

I've been a subscriber before, and there's a good chance I'll be a subscriber again.

rt66indierock's picture

I can't imagine auditioning any piece of equipment that isn't quiet enough to fully resolve a CD at low listening levels.

I never heard of Gordon Holt or John Atkinson for that matter until recently. I knew what pleased me emotionally at home and what I could listen to for long periods in the office without fatigue. Good reference albums and recordings volume matched have always gotten me where I wanted to go musically.

Surprisingly there was talk at t.h.e. Show about quantifying emotional involvement. Jason may have missed it.

I hope you come to RMAF.

John Atkinson's picture
rt66indierock wrote:
I never heard of Gordon Holt or John Atkinson for that matter until recently.

!!!

John Atkinson
Technical Editor, Stereophile

Bogolu Haranath's picture

Flummoxed? :-) .........

Bogolu Haranath's picture

Should he be sent to a DAEP school? :-) ..........

rt66indierock's picture

I’m the one with an Audio Alternative Education Program. First classes were held at t.h.e. Show in Long Beach California. Step one, go to Room 408 and tell me which pair of speakers is playing. The half that could tell the difference were sent to three rooms on the third floor and told to tell me the demo tricks being played on them in each room. I’m an easy grader, those who spotted at least one trick per room passed. This group would be worth teaching more about audio. Would you have passed?

Bogolu Haranath's picture

I'm afraid to take any tests ....... I would be a nervous wreck ........ Just ask any of my past teachers, if you can find them :-) ........

rt66indierock's picture

I like applying that kind of pressure. Just think of the fun I have with folks who read The Absolute Sound.

Bogolu Haranath's picture

I'm glad that you were not one of my teachers :-) .......

Bogolu Haranath's picture

I thought that TAS readers have 'golden ears' :-) ........

rt66indierock's picture

John, I first noticed you in a 2000 University of California Psychology paper I found in 2009 researching high resolution audio. You have been to Portland Oregon; I grew up there. I had access to information and people Lynn Olson wrote about in “The Soul of Sound.” I didn’t see any reason to pick a magazine sorry.

Bogolu Haranath's picture

Search for 'John Atkinson Editor of Stereophile' on Google : -) ..........

rt66indierock's picture

Why would I do a Google search for John Atkinson? I met him at RMAF 2016 and he has been more than generous with his time since then.

Bogolu Haranath's picture

Just for information, for anybody, if they are interested ........ There are videos and other detailed information posted online :-) .........

rt66indierock's picture

You can just ask him.

Bogolu Haranath's picture

My guess is, he is a man of few words and a lot of knowledge :-) ..........

Bogolu Haranath's picture

Next time you see him ask him, whether he likes his favorite Stone IPA or the supposed to be the top rated, Union Jack IPA :-) ........

rt66indierock's picture

He has an open invitation to visit me in the copper state and drink what ever he desires.

In your case my beer tests are harder than the audio tests. They involve field work.

Bogolu Haranath's picture

No thanks for another test ......... My favorite is America's number one selling beer :-) ..........

rt66indierock's picture

In that case you have to ask yourself three questions. Would Gordon Holt recommend Bud Lite? Would John Atkinson? Would Jim Austin?

Bogolu Haranath's picture

Gordon Holt, as I understand likes Martinis ........ JA1 likes IPA ........ Jim Austin, I think, likes Bourbon ...... I will go with the choice of the majority of the American beer drinkers :-) ........

rschryer's picture

...John Atkinson for that matter until recently."

You mean they're not the same guy?

Bogolu Haranath's picture

John Atkinson is an Australian actor ........ Gordon Holt was a former Scottish football player ...... Wikipedia :-) ........

ednazarko's picture

Where are those damn shoes? I took them off and now I can't find them!

RH's picture

I enjoyed this column Jim, and agreed with a bunch of it. I'm always
looking to balance the subjective and objective parts of the hobby.
I think another comment put it well: it's all subjective in the listening! (But...shouldn't be all subjective in the engineering).

As to this:

And speaking of uncertainty: We're not against statistically valid listening tests. We're just more aware than most of the challenges of doing them well and their tendency, as typically performed, toward missing effects that are subtle but real.

I can't help but mention again (I think the last time it was about JA's similar view), that that seems to beg the question. At least if one accepts or understands the justification for controlled (blind/double blind) testing in the first place.

If you take a difference between A and B to be real and audible, but can't detect this in controlled (e.g. blinded) conditions, how would you still decide it is real and audible? By what other method?

If you are ultimately wedded to going with subjective impressions no matter the results of the blind testing - "if blind testing doesn't validate my sighted impressions, then the blind testing was wrong!" - then this doesn't really seem to be having respect for blind testing. It would just obviate blind testing. When this is done it tends to leave the impression one is only paying lip service to the value of the scientific method, not truly being consistent with science.

I'm just trying to understand how you would square this because I very often see people using "I respect science BUT..." to make exceptions for every pet belief under the sun that doesn't hold up to rigorous scrutiny. Subjective experience is very powerful and everyone wants to drop the skeptical bar for their own most cherished experience or beliefs. I know I feel that pull.

(And I'm not arguing that the average audiophile/listener SHOULD start bothering with blind testing. But since you seemed to have raised the subject in your article, along with a particular claim...I'm guessing you don't mind comment on it)

"Plus, as the cliché goes, absence of evidence is not evidence of absence."

Ok, I apologize for being all picky here, but absence of evidence is evidence of absence. That's the case when the evidence one would expect if X were true is missing. We use this principle all the time.

Claim: there's a pink cat under my chair.

I look under my chair: no cat. Evidence of absence.

Claim: "Your house suffered a catastrophic all consuming fire today while you were at work.

I get home from work, and the home is entirely intact, clean, nothing burnt, no smell of smoke, no evidence of fire. Evidence the fire was absent.

Back to audio: if positive claims are made, positive evidence should be available.

I like the mix you have going at Stereophile!

Cheers!

Bogolu Haranath's picture

Lot of times listening impressions correlate very well with the measurements ....... That is the reason why Stereophile is grrrrreat : -) .............

RH's picture

Yes Bogolu, that's nice when it happens. Measurements correlate to some degree with what we hear; otherwise no speaker manufacturer would have any instruments to measure anything.

And I quite enjoy reading the subjective reviews in Stereophile. I've found them helpful sometimes in my own auditioning process.

I can't interpret speaker measurements with anything like the degree of experience John Atkinson has, so his comments in the measurements section are welcome. He also seems to have a keen ear in identifying issues, as a reviewer.

Ultimately, while I find the measurements section in stereophile to be educational, and enlightening to some degree, I don't purchase speakers on the measurements but on my own subjective impressions.
The Devore O series speakers were excoriated by some DIYers in these comments pages as being "wrong, wrong, wrong!" in every design parameter - nothing any educated speaker designer would build and the measurements "show it." And yet upon encountering those Devore speakers they became among my very favorite! If I looked strictly to measurements, and to the interpretation of those measurements by nay-sayers, I'd never have bothered to seek those speakers out. Rather, it was the subjective reviews I read that made me think "that's describing the type of sound I think I'd like." And that's what sent me seeking the Devores, and they remain on my list of possible speaker purchases.

So I still value subjective reviews. Which is one reason I still read Stereophile. (And why Stereophile is such a nice mix of subjective and objective - measurements)

Jim Austin's picture

RH, thanks for your thoughtful post.

There's a lot here, and I don't have time to respond to all of it, but I will make some comments.

Perhaps the hardest part of what I wrote is a deep conceptual shift. Those of us who are used to thinking as scientists often struggle to grasp that other perspectives can be just as legitimate. Just as likely to yield verifiable facts? No--but that is not the only valid pursuit. Humanists--and humans--have been gathering knowledge through other (less rigorous) means for millennia. Science is unique and special, but it's not the only game in town, and it cannot address all interesting questions, or anyway, probably shouldn't.

So that's the key conceptual shift: To realize that when it comes to music--even technologically mediated music--in a conversation between a committed scientist and a committed poet, the scientist doesn't automatically have the upper hand. There are dissertations to be written on this--probably have been written--but I have to end somewhere.

Now, "Absence of evidence is not evidence of absence" is actually an informal but pretty damn fine restatement of a principle of statistics. Here's a restatement from an AES paper by Les Leventhal: "Properly speaking, a statistical conclusion about H0 from a significance test cannot justify 'accepting' the scientific hypothesis that differences are inaudible." That article is: "How Conventional Statistical Analyses Can Prevent Finding Audible Differences in Listening Tests," Leventhal, presented at the October 1985 AES Convention in New York. Preprint 2275. Here, H0 is the null hypothesis, the hypothesis that the effect under test is inaudible. Leventhal is saying, in other words, that a failure to detect audibility is not evidence of inaudibility. It is merely a failure to detect audibility.

The reason I know about him is that he published an "article" in Stereophile, in the form of the world's longest letter. Highly recommended for those who know a little statistics and have unexamined faith in dbt.

If you are ultimately wedded to going with subjective impressions no matter the results of the blind testing - "if blind testing doesn't validate my sighted impressions, then the blind testing was wrong!" - then this doesn't really seem to be having respect for blind testing. It would just obviate blind testing. When this is done it tends to leave the impression one is only paying lip service to the value of the scientific method, not truly being consistent with science.

If blind-testing failed to validate my sighted impressions, then blind testing failed to validate my sighted impressions. It doesn't prove that my sighted impressions were wrong. There is only one way I can think of that a blind test could refute sited impressions. Let's say I insist, after listening, that two sources sound precisely alike. I conclude that they are the same--that they cannot be distinguished. Then someone else does that blind test and distinguishes between the two sources to impressive statistical certainty. THEN my sighted impressions have been refuted: It's audible after all. But it doesn't work the other way: If I hear something and then fail to pass the test, that doesn't mean I didn't hear it. It means that I failed to establish that I DID hear it. That's different. You can refute the null hypothesis. You cannot confirm it.

I really do believe in science--deeply. Over time, a preponderance of evidence can yield certain conclusions highly unlikely. When that happens, and it contradicts something I believe, I alter my beliefs. Many scientific-minded folks, though, are too uncritical--unscientifically uncritical--of statistical methods. We also tend to be arrogant in that we come to think that science is the only way to know things. Yet many people go through their whole lives without once practicing the scientific method--and yet they do know things.

Bogolu Haranath's picture

Mr.Austin ....... You are a perfect leader to keep Stereophile going forward :-) .......

RH's picture

I understand you don't have time to keep a conversation going and the following is not an attempt to drag you in to a debate. It's just my reaction to your own thoughtful post.

My response about "absence of evidence" wasn't meant as a rebuttal of the entire principle. As a matter of strict skepticism, we can not logically disprove any empirical proposition by empirical inquiry. That's the nature of empirical inquiry in terms of theory-building. As Quine cautioned, any result of testing a hypothesis can be rejected by the addition of another ad-hoc hypotheses. In other words, say we have the hypothesis: "If X chemical is present in water we will be able to detect X through our usual Y method."

Then, if you fail to find "X," instead of concluding X is not present, you can always add the hypothesis "Method Y is not sufficient to detect X."

Strictly speaking, any such additional hypothesis undermining a test results is possible. But (again as observed by Quine)...given we can always add another ad hoc hypothesis in this manner, this could be the undoing of science (and knowledge!). Which is why in our "web of belief" we get pragmatic about this stuff.

The problem is the dowser, the homeopath, the flat earther, the psychic, the energy healer (and on and on) all tend to share this trait of rejecting the usual empirical tests - which won't support their claims - by adducing ad hoc hypotheses which "invalidate" the tests and leave their powers unscathed.

And I start getting uncomfortable - epistemically speaking :) - when we in this hobby start to look more like we are in that camp. Where we find ways to rationalize away negative results in tests that we generally would not do so in other areas. For instance, my son has been involved recently in a double-blind, placebo-controlled study for a new allergy treatment. Neither we, nor the doctors who were also "blinded" in the experiment protested this procedure, either by saying "it's an insult to say our reports can't be trusted with the knowledge of which drug we are on. We aren't biased!" Nor did we invoke ad hoc hypotheses after the result came in, to explain away the results. Why? Because it would take ignoring all we seem to have learned about the role of human bias in distorting our empirical inferences, and why it's a good idea to control for this bias when possible. And that this problem of bias/inaccurate inferences expands far beyond the realm of medical research.

It just seems obviously special pleading to think my own hobby is exempt from the variable of bias. In fact, we can "know" that it is not exempt because - just like the cheap-wine-in-the-expensive-bottle experiments - you can show this variable of bias. As an example people have been fooled in to thinking A cable sounds different from B cable, when in fact no cables have actually been switched and they are identifying "differences" in the "sound" of the same cable!

So, yes, strictly speaking you are right that blind testing would only show "you have not been able to identify sonic differences under the conditions of the test."

But then, since that's the case in every possible experiment, we can't be paralyzed from drawing inferences from our experiments.
If we allowed ourselves to let our beliefs live in the skeptical gap, then nothing can ever be even informally disproved, and we join the dowsers, psychics, alternative medicine crowd.

It seems the pragmatic approach is to simply admit that if you do enough testing you should be able to draw provisional conclusions about an "absence." Or at least, an absence of evidence (which would in practical terms guide your expectations in the same way as "absence" would).

So, as we are being picky here about the inferences we can draw from blind testing, what I am not seeing in your reply is any suggestion of a "way out" of the mess that seems to arise when rejecting the implications of blind tests.

In other words: We KNOW that bias is a confounding variable in reaching our conclusions. Blind testing is as far as I know the most rigorous attempt to control for that variable. If we bring in the hypothesis that blind testing itself is unreliable, what is the alternative? Just letting those variables flourish in our conclusions?

That seems, as I've pointed out, to be special pleading just for the area of audio. And it is suspiciously just the type of special pleading we'd like to employ to safeguard our subjectively-derived beliefs - similar to how any number of fringe belief systems do the same.

I am not ruling out your point of view on this, or suggesting blind testing is infallible. It's all tough and often frustrating stuff.
But...these are the questions that come to mind when presented with the ways many audiophiles reject the implications of blind testing, and default to personal anecdote and subjectivism.

Cheers!

(And...I always have to be at pains to point out, the context of this discussion is one in which we take it we are talking about how to get maximal possible confidence in a conclusion, as when for instance doing science. This is NOT simultaneously the claim that the every day audiophile needs to, or should, be weighed down with the requirement of doing blind testing. No...we should all be free to buy what we want, for our own reasons).

ChrisS's picture

...on research methodology and hearing/perception.

You will have all the "answers".

Or you will keep asking the same questions again and again and again.

(Or put in another way, what you want, you will never find here.)

ChrisS's picture

...my music for me.

Nor anyone else for that matter.

Please take a college level course on research methodology and test design to know why no one uses DBT for audio reviewing.

No one.

And take a course on the physiology of hearing and perception while you are at it...

Anton's picture

We have some avid objectivists in our audio club here.

They have helped me solve the greatest mystery in all of Hi Fi: How does an objectivist choose his (it's always a he) Hi Fi gear?

By listening to it!

I know! LOL, right?

Did they double blind their listening and do instantaneous DBT switching prior to purchase? No. Did they buy the speakers with the best measurements? No. Do they eschew vinyl for only 24/96 digital? No. Do they talk about their favorite pressings of beloved LP's? Yes.

If it weren't for the internet, we'd all be subjectivists.

Then again, maybe our club's are phony objectivists and all the real objectivists just stay home, relax, and listen to digital audio on their well-measuring gear knowing there's no better measuring gear, so they would never join an audio club to chat about how audio gear sounds.

For the objectivists here: how did you pick out all your gear?

Bogolu Haranath's picture

By reading the reviews and measurements in Stereophile :-) ..........

Anton's picture

They shouldn't really need the review part, right?

(Cheers, Bogolu!)

Bogolu Haranath's picture

Read the comment below :-) ..........

Anton's picture

"Both" is fine, but then you are implying someone who claims to be an objectivist is simply lying or deluded.

RH's picture

"For the objectivists here: how did you pick out all your gear?"

The problem in answering to that call is that the term "objectivist," especially when used by a "subjectivist," tends to be larded with strawmen. As if raising issues of measurements, or worse, daring to defend the idea of blind testing at all, entails being some form of "Spock" who dismisses all subjective experience and pleasure, and thinks "everything sounds the same." (Yes, those very strawmen are raised quite often, here and elsewhere).

So my first question to you is: What do YOU mean by "objectivist?"

I don't know if I'll fit your criteria, but given I have some skepticism about some claims in high end audio, and will sometimes question the objective evidence for some audio claims, and even defend the (limited-by-practicality) utility of blind testing, I'm often assessed as an "objectivist" who thinks we should only buy gear on measurements or via blind testing.

Anyway...to your question. Personally I generally go by the "extraordinary claims require extraordinary evidence" heuristic.
If you tell me your house is lit via the power supplied by the local power company, I'm happy to accept that claim, given there's nothing controversial about it, and plenty of evidence that "power supplied to homes by electricity distribution companies" is a real thing.

If you tell me your house is lit by the energy provided by the perpetual motion machine in your basement...well...then I have every reason to be skeptical. Evidence at the level of personal claim or anecdote won't do.

Transplanting this heuristic to high end audio. For me, I'm happy to believe people hear differences between speakers, because there's objective, measurable evidence that speakers tend to alter the signal depending on their design. And much of that has been well correlated to what we actually hear.

We can still be fooled by sighted evaluations even with speakers, but as a practical matter we can't all go blind testing speakers and given the established plausibility that two speakers would sound different, I'm ok with going on my subjective evaluations. (Or believing other people hearing differences).

Things get a bit stickier turning to, say, amplifiers. At that point, among people with the requisite expertise - in electronics, and/or psychoacoustics etc - controversy arises as to the audible differences between some types of amplifiers (e.g. say between a competently designed SS amp operating within it's specs vs an expensive high end SS amp).

In that case I start to become more cautious about my conclusions, or about the claims made by sighted listening tests.

But...I use tube amps! To one branch of the "objectivist" side tube amps are a nonsense design. Why would anyone need a tube amp when cheaper and likely-to-be-more accurate solid state amps are available?

Well...because some of us like tube amps for various reasons. I like the aesthetic, the concept, and often the sound. I'm appealing to my subjectivism here.

If someone asks me "Have you blind tested your tube amps against solid state amps?" I'll admit: No, I haven't bothered to. My old CJ and Eico amps sure *seem* to me to produce a slightly different sound from my SS amps that I enjoy more. But I'll be the first to admit my evidence for this is only subjective and anecdotal, and for someone looking for a more confident, rigorous evidence I don't have it. And that's ok, I have no problem with that more "objectivist" person wanting that type of rigor. I get it. For me, I'm ok with going along with the fact that my tube amps are sufficiently "old school" design that even "hard objectivists" will generally tell me they are likely to interact with different speakers in ways that would change the signal. So even from an engineering stand point, it seems it's entirely plausible that my tube amps do "sound different."

Then we get down to claims about cable differences, especially AC cables and various audiophile tweaks. Some of these claims become even more controversial. When I see an almost uniformity from "objectivists" with engineering credentials saying "those technical claims are B.S and here's why!" I become skeptical of the claims made by the manufacturers of these products. I can't help but notice how the claims and anecdote-based "evidence" start to mirror pseudo-science, alternative medicine, and a myriad other dubious areas.

So...the skeptical/critical thinking hat goes firmly on. Now I'd prefer to see the type of evidence for those claims that *should be possible* given the claims. And which is readily available for other audio components like speakers and often amplifiers (and DACs...)

And if I'm dealing with an area I'm cautious and skeptical about, yes sometimes I will do my own blind testing. I've blind tested DACs/CD players (positive for identifying differences!), video cable (negative), high end AC cables (negative), and music servers (negative). I've found doing these occasional blind testing enlightening, sometimes fun, and it's saved me some money and some audiophile angst.

But what components I choose to put in my system and enjoy is of course entirely up to me, and no one should be telling me otherwise.
I had a strong subjective perception of an audible change with for instance a high end AC cable, which "went away" upon blind testing it. I chose to go with the blind test results and save money. But I could also have chosen to say to myself "you know what? Even though I can't detect this in my blind test, it just seems to change my perception when I put it in my system in a way that makes me enjoy my system more. So...I'll buy the thing."

That's just as much my right to make that subjective value as not buying it. BUT...I would not therefore claim to anyone else (or myself) that my subjective impression constitutes GOOD EVIDENCE for the claims made by the cable maker. If I or anyone else really wanted to get to the bottom of what is "really" happening, then it makes sense to get more rigorous, with measurements, controlled listener tests etc.

So...subjectivity...I'm all for it and ultimately use it in how I buy my stuff. But I would scale my *confidence* and *claims* about the the reality of what's happening on the quality of the evidence.

As to other "objectivists" you have to realize this hobby comprises a broad spectrum of views, a continuum, so you have to allow for nuance in people's positions and not try to box people in to false-dichotomy extremes.

Even among those who are very measurements-driven, there are differences. You WILL find audiophiles who do in fact look almost strictly to measurements in their buying decisions. And who do in fact employ blind testing, including ABX. Go to hyrdogenaudio forums, or Audio Science Review, and you'll see people sharing lots of blind testing for audio codecs, DACs, headphones, amplifiers etc.
Good on them, if that's how they want to do things.

Then there are those who will not buy strictly on measurements, but who will be guided by a decent knowledge of measurements to weed out speakers that aren't accurate in the way they want, and narrow their prospective purchases - ultimately deciding via listening which of those finalists to buy.

So, we audiophiles come in a broad spectrum - from "purely subjectivist" to "purely measurement/ABX-oriented objectivist" and everything in between.

Jim Austin's article above was trying to point out that you don't have to be in either extreme. A point I hope that doesn't go lost here.

Cheers!

Bogolu Haranath's picture

The signal is getting lost in the noise :-) ........

Bogolu Haranath's picture

Hi-Fi News does measurements of cables :-) .........

RH's picture

...and I replied in detail, I'm wondering if my reply was helpful or made sense to you?

JoethePop's picture

I read your magazine because of the “narrative listening journals” which I usually find highly interesting and well written, as well as the measurements with J.A.s assessment of them.
Yes, in the end, what matters most is what I hear and how, or if, it moves me; not the engineering behind it (though I find the engineering fascinating). The question always is, will I like what the reviewer likes? Will the sound have the same emotional impact on me that it did for him/her? This is where measurements can be invaluable. Because of the loss of so many brick/mortar stores, it is very difficult to audition and compare equipment these days. J.A.’s measurements, and perhaps most importantly his assessment of what they may mean to the sound produced, are the perfect balance to the subjective review.

Anton's picture

When you say: '"The question always is, will I like what the reviewer likes? Will the sound have the same emotional impact on me that it did for him/her? This is where measurements can be invaluable."

Can you explain?

I don't see the way that measurements would provide me with data about the emotional impact.

Apologies for my density!

Bogolu Haranath's picture

Let me give you one example ....... Recently reviewed BAT tube amplifier has high output impedance ...... Any such designed amplifier has very low 'damping factor' ....... hence, very little control over the loudspeaker transducers ....... One of the results is, bloated and boomy bass frequencies :-) ..........

Anton's picture

Wouldn't an objectivist simply see the measurements and skip the listening impressions?

You mentioned how it measured, so the subjective review would either be 'wrong' or unneccessary, it would seem to me.

Not trying to be silly, just that your review of the measurements said it all.

Bogolu Haranath's picture

HR who reviewed that BAT amplifier, mentions in his review that, that BAT amplifier did not work well with his Magnepan loudspeakers ....... Most of the Magnepans have 4 Ohm resistive load ........ That BAT amp has virtually no control over the Magnepan, planar magnetic transducers :-) ..........

Bogolu Haranath's picture

BTW ...... I could have guessed, how that BAT amp may sound with low impedance loudspeakers, without reading the review and, just looking at the measurements ........ It is still interesting to read the review, since HR reviews with several different components including speakers, amps, DACs etc. :-) ...........

Herb Reichert's picture

I read all the “well I guess you like the sound of distortion” comments associated with my review of the BAT VK65SE amplifier. And I laughed. Why? Because nobody seemed to notice that the BAT’s voltage amplifier and output stage used the exceedingly linear 6H30 and the 6C33B triodes. Both tubes generate plate curves that would be the envy of any transistor.

Likewise, no commenters noticed that the VK65SE employed only 3dB of global feedback! Therefore, the output impedance will naturally be higher and the damping factor will be lower than an amplifier with more feedback.

What do these complainers think? Victor Khomenko is incompetent? That he attended the prestigious Leningrad Polytechnic Institute and received an M.S. in physics and electronics, specializing in electronic emissions; plus spent his entire life in engineering; and could not figure how to achieve low THD or output impedance? Both “problems” are easily remedied with 20-30dB of global feedback. But at what musical-sonic price?

People said the same “I guess you like the sound of distortion” thing about First Watt SIT-3 designed by Nelson Pass. The SIT-3 is an ingenious study in simplicity and how to design a linear solid-state amplifier without feedback – by a proven master. But none of the commenters auditioned the amp or acknowledged the wizardry of its design. They judged the SIT-3’s effectiveness without ever using it for its intended purpose.

Please understand, all of today’s designers could easily make their amps measure and sound any way they want. Achieving good measurements is little more than high school math. Making an amplifier that will satisfy a broad range of users long-term is extremely difficult. And impossible if the designer has no clue what a high-quality amplifier should sound like.

Rest assured: Victor Khomenko and Nelson Pass measured and listened. But, if you only look at THD and damping factor - you will never understand what they achieved.

yin-yang herb

Bogolu Haranath's picture

Agreed ....... As you (HR) mentioned in the review of the BAT amp, matching with the right loudspeaker is important, whether it is a low damping factor amp or, a high damping factor amp :-) ..........

BTW ....... I'm not a 'Bear' ....... I checked with my family and friends, before posting this comment :-) ........

Herb Reichert's picture

my typo

herb

Bogolu Haranath's picture

No problem, thanks ........ I made that up anyway ......... I just checked with my bathroom mirror and made sure that I was not a bear, before posting that comment :-) ..........

Bogolu Haranath's picture

If HR wants to be adventurous, he could review the new Parasound Halo JC-5 stereo amp ....... It costs less than that BAT amp, BTW :-) .........

Bogolu Haranath's picture

Just one more suggestion, if I may ........ KEF R-3, $2,000/pair :-) .........

Bogolu Haranath's picture

As I suggested to you (HR) on an another forum, may be you (HR) could consider reviewing the Audiolab 6000A integrated amp? ........ 6000A was favorably reviewed in Hi-Fi News ........ 6000A has a very low output impedance of 0.01 to 0.03 Ohms ..... hence, a very high damping factor ........ Amazon sells Audiolab 6000A for $900 :-) ..........

rt66indierock's picture

Herb these complainers think you cannot identify a prestigious university. I’d be bitterly disappointed if the State University I graduated from was ranked as low as Victor Khomenko’s is in the world rankings. They also think you got the name of the university wrong. And finally, they are tired of your appeals to authority.

Bogolu Haranath's picture

The greatness of the University is determined by how much student loan debt you have :-) ..........

Bogolu Haranath's picture

For those who are interested ........ Stereophile has published an article written by RD, about Victor Khomenko, his partner Steve Bednarski and BAT :-) ..........

Ortofan's picture

... Victor Khomenko and Nelson Pass - to develop an amplifier that was subjectively "pleasant" or objectively "accurate"?

https://www.stereophile.com/content/manufacturers-comment-0

Could Nelson Pass design an amplifier that both measures as well as the Benchmark AHB2 and has the perceived sound quality of the SIT-3?

Kal Rubinson's picture

"I don't see the way that measurements would provide me with data about the emotional impact."

Really? I see many that infuriate me. :-)

JoethePop's picture

Measurements will never guarantee emotional impact, too many variables involved to claim that. They become invaluable to me (again including J.A.’s assessments) in avoiding speakers that I most likely will not like, regardless if the reviewer loved them.

sommovigo's picture

If the object of listening to live music is enjoyment, uplift, communion, eargasm ... there’s no reason to change standards when changing the ‘venue’ from live to home electronics. And as there is no reliable measurement for the experience of delights (among which are enumerated some, above) except as filtered through the human being in the throes of eargasm ... of what use to the listener, the music enthusiast is an FFT (etc), at all?

Measurements are useful to the designer, but seem fairly irrelevant once the component finds its final resting place in the listening room of the music lover.

“He who tastes, knows ...”

Robin Landseadel's picture

"Science requires us to have the courage to let our beautiful theories die public deaths at the hands of ugly facts. "

This is from an article from "Slate" about differences between Left and Right in politics. It applies to this thread as regards DBT. BTW, I'd like to see a DBT test, comparing the final track on an LP compared to the same track from its Digital source. In any case, here is an example of a study, Published in Science magazine, where an outside attempt to replicate that study failed to get the same result. Science magazine refused to post the rebuttal of the study they disseminated. I'm sure there's a lesson here to be learned that won't be:

https://slate.com/technology/2019/06/science-replication-conservatives-liberals-reacting-to-threats.html

RH's picture

I had been in dialogue with "ok" and all his comments have disappeared.

Server issues again?

Jim Austin's picture

OK deleted his own comments, replacing text with a period because our system does not allow people to remove their own comments. Rather than leave those empty comments up on the site, I deleted them.

Jim Austin, Editor
Stereophile

RH's picture

Ok, thanks.

An editor's work is never done!

ChrisS's picture

...having Stereophile help us enjoy our music.

rzr's picture

I would like proof that Stereophile is the most high-end audio magazine in the world. IMO, You are in the top 5. I subscribe to other audio mags and I wish Stereophile would adopt some of the other publications features to make it better. For example:
1) A couple of the other magazines have many more pages, many more reviews, and the magazines themselves have a 25% larger format, and 1 of them is $2 cheaper per issue.
2) As for reviews, you claim there is a checklist for a company to have their product reviewed, but last year, Art reviewed a friends product.
3) I still can't get over the Totem review you guys messed up years ago and then tried to make it right by saying Totem's most entry speaker was 1 of their best.
4) You have reviewers that are so biased on certain formats that they will never give an open-minded review of a product outside of their bias.For example, check out this months Letters claiming Fremer hates/dislikes SET amps. In response, Stereophile defended Fremer. The Letter could have gone much further stating Fremer hates anything DIGITAL! I will never read another Fremer review outside of an analog product.
5) A lot of space is wasted per issue. For example: in this months Follow-up, you did 2 follow ups on the Joseph and Revel speakers. You did a very good job on the Revel follow up and appropriate. The Joseph speaker that you did a follow up on was not the same speaker as the original, but the Revel review was. The Joseph speaker had new technology in it so in my mind, this should have been a new review of a new speaker.
6) Reviewers should have good/great audio rooms. A dining room or living room isn't a good room. If the reviewers room is on the small side, then they should review small systems, or components that are designed for their size room.
7) Measurements? Why do we need them? What is the SQ difference if 1 product has .004% distortion and the other .0004%? No other expensive HOBBY magazine that I know of has measurements: auto, photography, audio. Other mags dig into the technology of the product but they don't disect it. In the end, they let their SENSES (eyes, ears, feel) dictate if the product is good, not by a graph or scatter plot.
8) It's almost always unfair to say a component is the BEST you have heard in your system. Why? Because the last time you reviewed a like component, your reference system was probably different that what you have today.For example: if you reviewed a DAC 4 years ago, your system was probably comprised of different components than what you have today. Maybe a few cables were changed or maybe your amps/preamp were changed, or worse, maybe you moved to a place with a better room. To actually state for certain that 'X' component is the best I have heard in my system, your system and listening room must be exactly the same.
9) when will a reviewer produce a negative review? NEVER (except for the Totem fiasco). I'll read a review of a $50k-$100k speaker and there will be parts of the review that are negative, but in the end, it's a glowing review. The reviewer always tilts the review to be positive.
10) Why doesn't Stereophile produce a segment on each of the reviewers listening rooms? Its size, room treatments, dedicated circuits, flooring, wall materials? TAS and HIFI+ have done this for years and it gives the reader insight on why a system sounds the way it does in the review.
11) Stereophile should create a cook off between products: amp vs amp, DAC vs DAC for example. But, the format should be done similar to how the car magazines do it. When say Road & Track compares 3 sports cars, they have 20 different criteria they evaluate and give each criteria a rating. These magazines aren't afraid of giving a car a bad rating either in each category or as a whole, and even if they are comparing the best with the best, somebody will always come in second and third. By the way, these car reviews are not done blind, each car reviewer knows what car they are driving. Same goes for audio. A good reviewer should leave their bias's at home

X