I write this in a Seattle coffee bar—my flight home to New York has been canceled due to a snowstorm. As I try to put down these thoughts, I'm listening to the high-resolution masters of the April issue's "Recording of the Month," Sasha Matson's jazz opera Cooperstown, on my Pono player using Ultimate Ears UE18 in-ear monitors. I was in Seattle for Music Matters 10, held by retailer Definitive Audio, and this was my first road trip with the Pono since I reviewed it for the April issue. (Bruce Botnick and Charles Hansen comment on that review elsewhere in this issue.)
I realize now that, in my review, I did not say enough about the compelling nature of music played through the Pono. During the long flight out to the Northwest, and now as I write these words, my attention keeps being drawn to the music in a way that rarely happens with my iPod Classic. This happens not only with hi-rez PCM files (and DSD files, which, with the release of firmware v.1.0.5, the Pono can now play), but with CD rips and even MP3s. Perhaps the best way to characterize the Pono player is to say that for $399, you get a D/A processor almost as good as Ayre Acoustics' QB-9 ($3250), with a 128GB hi-rez media player thrown in for free.
So it was with not a little astonishment that, while writing about the Pono, I read negative reviews of this little gem in the mainstream press. David Pogue, late of the New York Times, wrote for Yahoo.com's Tech pages: "The Emperor Has No Clothes . . . Neil Young and the believers in high-res audio aren't fools, and their hearts are in the right place. But Pono's statement that 'Everyone who's ever heard PonoMusic will tell you that the difference is surprising and dramatic' is baloney" (footnote 1). "Neil Young's PonoPlayer sounds no better than an iPhone—no matter what the audiophiles say," wrote Seth Stevenson for Slate.
Do these writers really not grasp what Pono is about? And do they really hear no improvement in sound quality—not just with the Pono player in particular, but with high-resolution audio files in general?
As I read the reviews, I was struck by just how inadequate they were. I have been involved in audio for all of my 45-year working life—as a professional musician, a recording engineer, a record producer, a magazine editor, and a reviewer—yet I am well aware of how much I still have to learn. I don't understand how the mainstream media will use experts on sports to write about sports, experts on the visual arts to write about painting, experts on music to review concerts, etc.—but when it comes to audio, they commission articles from generalist technical writers.
Their listening tests were, in my opinion, incompetently designed, using naïve listeners, and the results didn't support the conclusions drawn. For example, some of the mainstream reviewers compared the Pono player with smartphones. However, the reviewers were confusing a test for preference, which is what they had performed, with a test for difference (footnote 2). As a result and not surprisingly, the listeners tended to prefer the sound with which they were most familiar; something I explored in my 2011 Richard C. Heyser Memorial Lecture to the Audio Engineering Society. And as Michael Lavorgna wrote, "people who aren't all that interested in the sound quality of the music they listen to are not all that interested in the sound quality of the music they listen to."
So what happened? Neil Young must bear part of the blame. As an outspoken critic of CD-quality sound and of MP3, Young was sticking a "kick me" sign on his back for these writers, none of whom had been bothered by the poor quality of these lower-resolution formats.
But also to be taken into account is the herd mentality of the typical mainstream writer. Fred Kaplan (footnote 3) wrote me that such writers always make fun of expensive hi-fi gear, while accepting as normal expensive cars, wines, auction prices for paintings, etc. Such writers don't question the market for $4000 place settings, $40,000 wristwatches, $400,000 cars, or $4 million houses, because such extravagance is supported by the advertising that appears in their publications. No, there is not a direct connection between ad revenue and what writers write, but the advertising for luxury goods other than audio confers on high prices a societal legitimacy—and no such legitimacy has been conferred on audio. Even with a $400 audio product—$400!—a mainstream commentator would be going into unexplored territory if he or she wrote positively about it, no matter how high the actual sound quality (footnote 4).
But I think the issue goes deeper than this. Mainstream technical writers tend to practice access journalism; in short, they describe what people say or do: Apple releases a new iPhone, Sony releases a 4k TV, Linksys releases a new router. By contrast, in accountability journalism, the writer judges what has been said or done. Audio magazines practice both kinds of journalism; show reports, for example, are access journalism, while reviews are accountability journalism. The first is straightforward; the second is not.
Much of what passes for accountability journalism on the Web is actually access journalism: a product is described and its functionality discussed, but it is neither judged nor put in its market context. The mainstream press's coverage of Pono is an example of access journalists venturing into accountability journalism without being equipped with the necessary tools (footnote 4). The sad reality is that, in my opinion, such writers just don't know enough to know how little they know about audio, or how much more they need to know in order to be able to write a responsible review of a genuinely superb audio product—such as the Pono Player.—John Atkinson
Footnote 1: Links to some of these reviews can be found in Michael Lavorgna's excellent essay on AudioStream. Footnote 2: The writers seemed unaware that there is an entire scientific discipline devoted to designing blind tests that actually produce meaningful results, that such blind tests are complicated and time-consuming when the differences are small but nonetheless important. They seem to believe that merely the fact that a test is blind confers legitimacy on the results, which is just plain incorrect.
Footnote 3: Fred wrote an excellent defense of audiophiles and high-end audio for Slate.com.
Footnote 4: Jason Stoddard of Schiit Audio wrote an excellent essay on the mainstream's rejection of Pono.
Footnote 1: Links to some of these reviews can be found in Michael Lavorgna's excellent essay on AudioStream. Footnote 2: The writers seemed unaware that there is an entire scientific discipline devoted to designing blind tests that actually produce meaningful results, that such blind tests are complicated and time-consuming when the differences are small but nonetheless important. They seem to believe that merely the fact that a test is blind confers legitimacy on the results, which is just plain incorrect.
