The Great Wall of China

You could sense the frustration in Keith Pray's e-mail. "We are on the same team. I have always respected your wishes and will continue to do so," he had written me. At the request of a possible advertiser, Stereophile's publisher had asked me a question about something appearing in the issue of Stereophile we were preparing. I had responded that not only would I not give him an answer, I felt it inappropriate for him to ask.

I wouldn't usually mention an internal and inconsequential matter like this if it weren't for the fact that the day Keith and I had our altercation, the news broke about another magazine editor, who doesn't appear to share my reticence about crossing the "Chinese Wall" between a magazine's editorial and advertising sides. In fact, according to an August 6, 2006 report in the Orange Country Register, the longtime editor of Fanfare, Joel Flegler, doesn't just cross the wall, he leaps over it.

In a July 19 blog, the OCR's classical music reporter, Tim Mangan, had reprinted a letter Flegler had sent to an independent record company that was inquiring about review coverage in Fanfare. Flegler had proposed a quid pro quo: that while his magazine might review the CD in any case, a way to guarantee publication would be for the company to buy advertising space. Flegler's letter included a list of what ads of various sizes would cost and suggested that the more money the record company spent on advertising in Fanfare, the more editorial about them could be published—such as an interview with one of their artists.

It is important to note that Flegler had not promised the record company positive coverage, something confirmed by a couple of Fanfare reviewers in the blog discussion. But without the company buying advertising in Fanfare, Flegler would not give reviewing the CD "top priority" in the publication queue. And, as Mangan said, "So what? An editorial decision had already been made, before the review."

I agree with Mangan that such a direct connection between editorial coverage and the purchase of advertising renders a magazine's content "worthless." I am ashamed that Flegler and I both put "magazine editor" down on our job descriptions.

I wouldn't usually write about a matter such as this. I feel that magazines should not take public potshots at one another. But I have received questions from Stereophile readers asking about this practice, suggesting that if Fanfare's editor has no qualms about trading review coverage for advertising, then how can they be sure I don't? Such accusations are hardly new. In a Manufacturers' Comments" letter published in our December 1989 issue, responding to a 1989 review (recently posted in our free online archives), Waveform's JohnÖtvös had whined that the negative comments we had made about the sound of his speaker and the measured problems I had found owed more to the fact that Waveform wasn't an advertiser than to anything real. Which was as much BS then as it would be now (footnote 1).

I have written many times in Stereophile about the separation between our editorial decisions and the presence or absence of advertising. (See, for example, my March 1996 essay, "The Great Wall of China.") I assure you that that separation has survived many management changes at Stereophile. "We are on the same team," Keith had reminded me in his e-mail, and in a wider sense he is correct. But I play primarily for the editorial team, which is, I believe, why on October 2 I am about to celebrate 24 years, without a break, of successfully editing audio magazines. [This was written seven years ago at the time of reposting to www.stereophile.com; on October 2, 2013, I will be celebrating the the start of my 32nd year at the editorial helm of an audio magazine.—JA]

But don't take my word for that separation. Look at the evidence. If I gave Flegler's "top priority" to reviews of products made or distributed by advertisers, that would be readily evident from an analysis of our coverage. Here is such an analysis:

I added up how many products had received more than a nominal mention in Stereophileie, products that had been written about in a formal equipment report, a Follow-Up report, or in a column by Sam Tellig, Mikey Fremer, Art Dudley, Kal Rubinson, or John Marks—between October 2005 and April 2006. I split the list into current advertisers in Stereophile and non-advertisers.

The results: 42 products reviewed were from advertisers, 48 from non-advertisers. So if anything, we favor non-advertisers when choosing what to write about.

However, if there are very many more non-advertisers than advertisers, and if we chose a representative selection of audio products from both groups, this means that reviews of products from advertisers should be much fewer in number than reviews of products from non-advertisers, instead of only slightly fewer. How do these statistics look when examined in that light?

We are in the processing of compiling our 2007 Buyer's Guide, due to be published in November 2006, which excludes by intention audio products intended for home theater and architectural use that we don't cover in the monthly magazine. The database we have compiled over the past four years of companies that manufacture products eligible for review in Stereophile lists 363 separate brands. Inevitably but regrettably, some brands get omitted. So let's say that the set of brands eligible for review in Stereophile is 370 at maximum, but probably less, given that we list brands in the Buyer's Guide that don't have the necessary five dealers to qualify for a review.

I don't track advertising in Stereophile, so I asked Keith Pray for a list of active advertisers; ie, brands that had been advertised in Stereophile in the past 18 months. That total is 120, not including brands that had advertised only once, which would require rather a deeper dip into the database than he had time for. Let's assume that including those onetime advertisers would increase the total number of brands currently advertising in Stereophile to 150.

The results:

Number of advertisers as proportion of eligible brands: 40%
Proportion of advertised brands featured in Stereophile reviews: 47%
Number of non-advertisers as proportion of eligible brands: 60%
Proportion of non-advertised brands featured in Stereophile reviews: 53%

So, in the worst case, there is a slight bias in favor of advertisers, in that instead of 40% of the products we choose for review coming from advertisers, the proportion is actually 47%. But you don't have to accuse me of "pulling a Flegler" to explain the 7% disparity. There are many other factors involved. The set of companies that advertise in Stereophile has high correlations with: the set of companies that have a high profile in the marketplace; the set of companies that have been in existence for 10 or more years; and the set of companies whose products have been favorably reviewed previously, not just in Stereophile but in all audio magazines, and are thus more successful in the long term. Such companies also exhibit more regularly at Consumer Electronics Shows and have a higher profile at dealers, and are thus more likely to have their products auditioned by Stereophile's reviewers when they are trying to decide what to write about in future issues.

The effects of these factors are impossible to predict, but I believe it is fair to point out that they will diminish the slight correlation noted above. Which, again, arose from looking at the raw data in the worst possible light for Stereophile.

Normal service can now resume, now that I have shown that "The Great Wall of China" is in a state of good repair at this magazine.


Footnote 1: This false meme is as alive in 2013 as it was in 1989—as I was preparing this essay for republication, I was contacted by a manufacturer concerned that as he was no longer an advertiser, we would be dropping his products from "Recommended Components." I assured him that whether or not we recommended his products had no connection with whether or not he advertised.—John Atkinson

COMMENTS
dalethorn's picture

Much nodding of head...

andy_c's picture

Much as I dislike Salvatore, he nonetheless hits the nail on the head with his "Rules for Audio Reviewers". Rule 8 is as follows:

8. The more corrupt your magazine is, the more you shall proclaim your honesty.

dalethorn's picture

I can visualize the corrupt proclaiming their honesty while concealing the details, which doesn't apply here.

Michael Fremer's picture

So what you're saying Andy, is that when you are attacked and accused of being corrupt by the likes of Arthur Salvatore, what you're supposed to do is just shut up and take it because to proclaim your innocence is to prove your guilt?

You'd be right at home as would Salvatore as a Salem Witch Trial judge.

Sorry bud, turning the other cheek is good for Jesus but not for me. Arthur Salvatore's brain is filled with paranoid vipers that see "corruption" at every turn. In his mind, to disagree with an Arthur Salvatore conclusion is to be corrupt. That much is obvious in virtually all of his opinions, whether about equipment or used records or whatever. 

One reason I went after him years ago (probably too strongly but what the hell, he deserved it) was when he accused ME of being corrupt. He was critical of a review I wrote of a Crown Jewel SE cartridge. Now that's FINE. Don't agree with the review? Fine. Say so. 

But what Salvatore's paranoid vipers concocted out of thin air and made him put before the public to see was a totally fabricated story in which the reason I  wrote the positive review, while failing to mention that the cartridge was IDENTICAL to a particular model Shelter cartridge, was because I was doing "a favor" for "my friend" Steve Klein, who imported and distributed the cartridge.

There are probably more dark accusations made by Salvatore about me in his concocted story, but I particularly remember that one because at the time I had met Steve Klein exactly ONCE in my life when he visited to drop off a review sample of the Simon Yorke S7 turntable and one Klein's Sounds of Silence air isolation systems.

Klein was not and is not someone I consider "my friend". He's not an enemy, I like him, the few times we've actually met but he's not someone I consider or then considered by friend. But Salvatore just couldn't help himself and posted his delusion in order to explain something he didn't like that I'd written.

But beyond that, the Crown Jewel SE, while built by the same people who build Shelter cartridges, was NOT the same cartridge as the Shelter cartridge he claimed was identical. It was a far more sophisticated design using more expensive parts.

So Andy if you believe Arthur Salvatore's paranoia driven Kafka-esque notion that the more a magazine proclaims its honesty, the more corrupt it actually is you are welcome to live in Salvatore's paranoid and putrid world. 

Not me!

JohnnyR's picture

Damage control in full operation even as we speak looks like.  Talking about what they do so well and "thank goodness we aren't like other magazines or reviewers" while ignoring what they don't do at all. Black hats trying to pretend to be wearing white hats. I'm sure no one will be angry with this opinon cool

jimtavegia's picture

I hardly think that applies to Stereophile, but more to our current political climate. I see more liars in suits than I do open-collar shirts. But, It is clear that most people seem to like being lied to, or at least mis-led...with a smile. Graft and corruption are nothing new. I just don't see it at Stereophile. 

I remember the issue with a Totem speaker a while back that seems to be a very honest mistake and unfortunate circumstance for all involved. Who of us would know if the reason that someone did not like product X was the fact that product X was "broken" in some manner that no one knew about? It may be that as a speaker or piece of gear makes the "reviewer rounds" it should always go back to the manufacturer first to be checked out before moving on to the next reviewer. 

I think about JA's new Sony speaker review in the newest issue and the blown tweeters. The honesty I have seen from the pages of Stereophile is probably the highest I've read in  anything in print since 2008. 

Since most of us have lost involvement in local brick and mortar stores due to their closing, auditioning gear is getting near impossible. 99% of the population do not attend regional audio shows to audition gear. Without magazines like Stereophile most of us would not know anything about the quality or sound of audio gear. 
 

Would the vinyl resurrgence have happened without the likes of Stereophile and the internet? Finding out about a lp pressing plant in Kansas, of all places, would be left unknown. It was never mentioned in our local Atlanta news paper. 

If I had a life outside of teaching Math in HS, I probably wouldn't look forward to my next issue of "Phile" as much as I do. Music is a huge part of my life and without 
Stereophile I would have made gear mistakes and missed out on knowing some great music to buy. 

You should be much more weary of your politicians than Stereophile's hidden agenda. 

Michael Fremer's picture

I received a lot of hate mail after my Ortofon Anna review ran. I had honestly revealed that the first sample of the $8000 plus cartridge was defective because the adhesive turned out to not be compatible with the elastomer. The second sample had another problem that I also revealed. This was a brand new model and as with many new products problems sometimes occur that require sorting out after its introduction.

The third sample was and is fine and I recommended it highly. It's a superb, technologically advanced cartridge.

Yet I was accused by more than one obtuse reader of being "in Ortofon's back pocket" for recommending it.

Were I really in Ortofon's "back pocket" I had to remind these dummies, I simply wouldn't have written about ANY of the problems I encountered with the first two samples. 

FastEddy's picture

I agree with Rule 8's comments above.  But I find some amusement in the cranks and nuts that post on this and other audio forums.  They always bring to mind Bill Maher's movie Religulous and the part where he is talking to the crazy Rabbi about holocaust deniers.

Michael Fremer's picture

I think if you agree with Rule 8, you are among the cranks and nuts. See the Salem Witch Trials for instance.

FastEddy's picture

I meant I agree with Mr. Tavegia's comments under the heading "Rule 8".  The concept of the so-called Rule 8 is absurd but very common.  Sinclair Lewis wrote it as 'it is difficult to get a man to understand a thing when his salary depends on his not understanding it'.  

Bill Leebens's picture

"As soon as you crack your knuckles and open up a comments page, you just canceled your subscription to being a good person."

This doesn't apply to you, John--how you maintain your cool, relative to this endless stream of bullshit, is beyond me.

John Atkinson's picture

Bill Leebens wrote:
how you maintain your cool, relative to this endless stream of bullshit, is beyond me.

A natural sedative, Bill, available in bottles sourced from the Sierra Nevada Brewing Company. That and my favorite music playing as I type. And responses like that above from Jim Tavegia, whom I had the pleasure of meeting at a dealer evening in Atlanta last April.

John Atkinson

Editor, Stereophile

jimtavegia's picture

I find it sad that issues like this seem to come around every so often when I see no reason for it. That dishonest kind of behavior is usually left for ones out for a quick buck, and it is clear that Stereophile, you John, and your writing staff seem to want to be around for another 20+ years and hundreds of more issues. 

I am sure that those whose stereo equipment purchases are in 6 figures are glad that Stereophile is around and has helped light the path to the better gear. The fact that you also cover gear that only costs  hundreds should make all aware that the only agenda is to see that as many manufacturing voices as possible can be heard and applaud the ones who do well with their designs. 

Paranoia is a sad thing.  I'm glad I don't suffer from it. I also eagerly await Mikey's revealing of his phono cartridge test. My hearing flaws WILL be revealed. 

Thanks for coming to the event in Atlanta at Audio Alternative. It was great fun and I strongly recommend attending an event in someone's home area. 

volvic's picture

I feel I must comment on discussion; 

 

Loved/Appreciated Fremer for telling us that 2 mega expensive Ortofons had a quality control issue

Enjoyed reading the Sony review from JA's article in this month's Stereophile including the aforementioned blown tweeter.  

Have never had a whiff of whatever is being discussed above and from others, and will renew my subscription when time comes.  Love getting the mag and the connection I feel with its writers every month.  

As the late great Karajan once said "the rest is gas light"

John Atkinson's picture

volvic wrote:
As the late great Karajan once said "the rest is gas light"

Now that brings back memories. I attended the Salzburg, Austria press conference where the final specification of the CD was issued and von Karajan uttered those imortal words.

John Atkinson

Editor, Stereophile

Bill Leebens's picture

Y'know, John, CDs do sound better when read with a gas laser than with a solid-state one. The Maestro was on to something.

 

Just sayin'.

volvic's picture

Now that brings back memories. I attended the Salzburg, Austria press conference where the final specification of the CD was issued and von Karajan uttered those immortal words. - JA

Lucky you that was a historic moment in hi-fi history.  If I recall Akio Morita was there and the CD player was a large pre-production model.  Many moons ago over a drink, Jon Vikers told me that Karajan was able to tell who of his musicians was subpar during performances in Salzburg, he called out Mr. Vikers one night after a performance telling him he was only at 75% of what he normall was.  Mr. Vikers told me that he wasn't feeling well and took a long nap and woke up not feeling any better before the performance.  It is strange that such a keen ear went on record always preferring 16 bit technology over his older analogue recordings and saying it could not get any better.  I wonder if he were around today he would most certainly be re-recording his favourite repertoire in SACD.  

jimtavegia's picture

I had a band director that I used to record his high school fall and spring concerts and after the 3rd year he said that he would rather not waste my time has he felt  the performances were so bad he did not wish to relive them. 

That also happened at the saxophone symposium with 50 plus high school players about to enter our local college after the 2nd year of the Symposium. The Professor there said that the quality of the players coming into college is getter worse every year. I'm sure that at the major universities that is NOT the case I hope. 

Gene T's picture

To imagine that there's never been, or might never be, a review of a commercial recording, an audio component or an industry/consumer trade show, published at one time or another in any hi-fi magazine, which couldn't in some way have been related to advertising bought, discount pricing given, or other considerations rendered to a magazine's editors, staff and contributors, requires a certain suspension of logic and common sense.

On the other hand, to hint that anyone who reviews commercial recordings, audio components, etc, might be likely to suffer from such corruption simply because oneself is convinced -- even while lacking any evidence beyond opinion and conjecture -- that such unethical behavior rampantly exists, is equally loopy.

It is left to the individual reader to make his/her own judgements regarding consistence in the veracity of an individual publication's review content. 

Personally, I've never read anything in Stereophile that suggested to me that I was being BS'd by a reviewer because he or she was somehow obligated to help pitch the product under review. I've read the occasional review herein that struck me as a bit incomplete or somewhat confusing, or the odd one that seemed to be of slightly lessor standards than the average of what I've come to expect over the years, but never have I come away with the impression that a Stereophile reviewer was in the manufacturer's or advertiser's "pocket."

remlab's picture

...regarding this issue is when a reviewer purchases a $200,000 loudspeaker.  Yikes! 

NRG's picture

J.A.,

Regarding:

"The results: 42 products reviewed were from advertisers, 48 from non-advertisers."

Would you please add links to the 90 articles referenced, categorized by "advertisers" and "non-advertisers"?

I think readers will be interested to see:

1. Which products were "recommended" vs. not?

2. If non-advertisers products were compared less favorably to advertiser products.

Keep up the good work.  I assume the links/articles will support claims of impartiality, and we'll all enjoy reading them.

Best Regards

John Atkinson's picture

Quote:
please add links to the 90 articles referenced, categorized by "advertisers" and "non-advertisers"?

I am afraid that that is way too much work for me to do to satisfy someone else's curiosity :-)

Quote:
I think readers will be interested to see:

1. Which products were "recommended" vs. not?

2. If non-advertisers products were compared less favorably to advertiser products.

Someone on the Audio Asylum has done this analysis. His conclusion was that there was no difference between products from advertisers and from non-advertisers, when it comes to recommendations in Stereophile.

John Atkinson

Editor, Stereophile

Rick Tomaszewicz's picture

Reading the comments on your article, I can't help but think it'd be more fun to see flame wars erupt over music and performances, rather than over gear and accusations of conflicts of interest.  But then again, I do keep reading this site and Mikey's every day.

7ryder's picture

I'm more interested in understanding why Stereophile continues to dismiss manufacturers who sell directly to customers as not worthy of a formal review because they don't pass the 5 dealer minimum. Seems to me that these Internet direct companies have a much greater reach and greater ability to get their products into readers' hands (or potential readers' hands) than a manufacturer that has 5 dealers in all of the USA. 

Frankly this policy is so last century and to the cynics among us, could lead them to believe that the only reason this policy exists is because folks like JA and MF get invited to many of the brick & mortar open houses. 

Now I would hate to see the surviving brick & mortar dealers go out of business because they do serve a purpose, but to ignore companies such as Wyred4Sound and others that produce some pretty great sounding products just because they prefer a different distribution model is doing your readers a disservice. Additionally, it drives potential readers/subscribers (such as myself) to other magazines/websites to read about these products and I'd think you would like to get as many eyeballs as possible looking at either your printed or web based media. 

I think the audiophile tent is big enough for Stereophile to let traditional manufacturers in as well as those that sell Internet direct and review products from both. Exposing your readers to more manufacturers is good not only for your readers, but it also helps get new companies established and will ultimately lead to a stronger audio community. 

 

 

 

John Atkinson's picture

Quote:
I'm more interested in understanding why Stereophile continues to dismiss manufacturers who sell directly to customers as not worthy of a formal review because they don't pass the 5 dealer minimum.

As Stereophile does give full reviews to products that are sold direct, your criticism is without foundation, I am afraid.

See www.stereophile.com/asweseeit/307awsi/index.html.

John Atkinson

Editor, Stereophile

 

 

 

 
7ryder's picture

I guess I was wrong to infer that the 5 dealer rule was still in effect by following the link in your article.  Certainly, though, you can understand my confustion since your article not only states "necessary 5 dealers" which is underlined and in bold AND with a link to the 5 dealer rule (and not to the link you provided in your reply to me which states a revised review policy).

There are many musicians that get great results producing other artists' recordings, but when they insist on producing their own recordings, they often get mixed, if not, disappointing results.  I always wonder if the end result would have been better had someone else had produced their recording.  

Perhaps the same is true of editors who write and then edit their own work.  With apologies to Alan Moore - "who edits the editors?"

John Atkinson's picture

7ryder wrote:
I guess I was wrong to infer that the 5 dealer rule was still in effect by following the link in your article.

I don't think you understood what you read. Yes, I wrote in the linked article (www.stereophile.com/asweseeit/307awsi/index.html), "products that are distributed through conventional retail channels must be available at five or more US retail outlets (www.stereophile.com/asweseeit/746)."

But the key phrase is "distributed through conventional retail channels." At the other article linked to in this text, I wrote that "I make exceptions for mail-order or Web-based companies." I elaborate on this in the first article: "It is the editor's decision whether a product distributed by mail-order or via the Web qualifies for review. However, at minimum the company must have a formal US presence and must offer a 30-day, money-back refund policy."

I fail to grasp what is unclear about this policy, which has remained consistent for almost 25 years. 

John Atkinson
Editor, Stereophile

Synthetic's picture

It's unfortunate that you still face character accusations.You really do buyers an enormous service by publishing comprehensive equipment measurements that loudspeaker manufacturers in particular have failed to provide. If there is a slight product selection bias towards advertisers, at least you help keep them accountable.  

X