Huh?

Quote:
Huh?

Policy? We don't need no stinkin' policy. We don't have to show anyone no stinkin' policy.

Thanks goodness for Kal's comments; otherwise, the review would have been useless other than to those that wanted to know it's fun to use. Importantly to me, Kal confirmed to me that it's DAC was bottom of the heap type stuff, but if you ran the digital output through a decent DAC then it became a true contender. Now that I have the great Playback Designs DAC in my MPS-5 player, the Sooloos is an actual option for me.

You have to take the reviews in Stereophile as a whole and read between the lines most times. That's no news and it's typical of other mags. The primary reviewer wasn't really equipped with enough high quality references to say anything useful about the sound vs. the very best digital sources. It's likely that JA realized this and pulled in Kal. So what? At least they ended up with a comphrehensive review and not just some "oh it was fun to play with" kind of crap.

Great stuff, IMHO. I understand JA's need to rescue the review. It's an important review of an expensive set of equipment and he couldn't let it lay with only comparisons to Denon and Benchmark DACs, which are fine pieces of equipment in their price ranges, but not meant to compete in the $10,000 and up league.

Dave

Quote:
In the case of the Sooloos review, it would seem that JI was aware of JA

Quote:
The primary reviewer wasn't really equipped with enough high quality references to say anything useful about the sound vs. the very best digital sources. It's likely that JA realized this and pulled in Kal.

This is upside-down. Kal was originally going to do the full review, but I felt that Jon Iverson, who is an IT professional, would be better suited to write about the network and interface aspects of the Sooloos, which is really where its innovation lies.

John Atkinson
Editor, Stereophile

I think the Sooloos and other all-in-one solutions are pretty much for well-heeled technophobes. You could get the new HP touch screen pc for about $1k, a couple of Apple Airports, a DAC or two of choice and you are truly good to go for under $2k (depending on your DAC choice of course). But even an easy setup like that would require a little networking knowledge so I think most people would spend the extra money for someone to do it for them. Doesn't make economic sense to me, since you could get a local PC dude to set it all up for a couple hundred bucks, but then again it's not all packaged and marketed that way.

The existence of the Sooloos, and the fact that the Absolute Sound and Stereophile both wrote sizable reviews of it, points to how frustrating tagging and organizing ripped music can be.

Quote:
HUH?

I didn't mean to be opaque. Perhaps I was.

The sole appeal to the Sooloos is the interface and the fact that someone else does the organizational work.

(The thread has drifted away from the original question as this question has already been answered.)

Quote:

Quote:
Reviewers don't see the measurements before they submit their review text to me for publication. Otherwise, the danger is that they will then hear what they read.

That's what I recall the policy was. I would think it would apply to comparing notes with another reviewer as well.

The question hasn't come up before, but currently the policy allows this.

Quote:
I guess I need to brush up on my reading comprehension skills. I don

I don't see where anyone could read the whole review and get the idea that the DAC was anything better the low end of the heap. All it took was a Benchmark to improve on it. At 10,000-buck I'd expect a little more effort in the sonics department. Still, knowing that my own DAC will fix that, makes it a possibility.

Except for my discomfort with the DAC I think that the pricing is the region of fair. Packaging ain't cheap and getting it done right takes a lot of development. Given the low volume expectations of this device I think that were taking a risk even at the original price point. As more dealers sign on and the volume increases we are likely to see price declines, particularly as more competition enters the pool.

Dave

Quote:

Quote:
Jon Iverson was made aware of the measured data _after_ he had written the review when he was sent a preprint of the complete review for proofing. If I remember correctly, he wanted to add this text to underline the fact that Sooloos's use of an aftermarket DAC didn't negatively impact its performance.

LOL. Why have a policy if you

John, you don't need to explain your reviewing process. The magazine's reputation speaks for itself. Just stay away from the MSNBC political editorials unless you're willing to review the equipment the CIA uses.

Just kidding big guy. Don't accept any equipment from the CIA for review. Really, I mean it.

Quote:
John, you don't need to explain your reviewing process. The magazine's reputation speaks for itself.

Thanks, but I do understand rgibran's concern: that while reviewers don't see the measurements before they write their findings, they do see them afterward. So what is to prevent them from rewriting their review to better conform to the measured performance.

As I said, that doesn't happen. What I am interested in is not only the correlation between what they hear and what I measure but when there is a _lack_ of correlation. That gives me feedback on the usefulness of the measurements I perform.

John Atkinson
Editor, Stereophile

Great thread and great responses from JA, as far as I'm concerned. This truly highlights the usefulness of this forum.

Dave

Quote:
What I am interested in is not only the correlation between what they hear and what I measure but when there is a _lack_ of correlation. That gives me feedback on the usefulness of the measurements I perform.

John Atkinson
Editor, Stereophile

Feedback on the usefulness of the measurements or the reviewer's ability to hear?

Quote:

Quote:
What I am interested in is not only the correlation between what they hear and what I measure but when there is a _lack_ of correlation. That gives me feedback on the usefulness of the measurements I perform.

Feedback on the usefulness of the measurements or the reviewer's ability to hear?

Both. For example, I may measure a strong vibrational resonance at a specific point on a loudspeaker's side panel, but the reviewer didn't comment on any midrange congestion or coloration that could result from that resonance. There are two possibilities: 1) the reviewer should have heard the effect of the resonance but didn't; 2) the problem measures worse than it sounds, perhaps because the affected area is small so doesn't couple to the air to a any great extent, or perhaps because its acoustic output is canceled by opposite-polarity radiation elsewhere on the panel, or because its Q is so high that the resonance is never fully excited with Western-scale music.

My hat's off to my review team, who offer up their subjective evaluations in the full knowledge that my subsequent measurements may expose their shortcomings. Overall, I think they do a remarkable job.

John Atkinson
Editor, Stereophile

Quote:
My hat's off to my review team, who offer up their subjective evaluations in the full knowledge that my subsequent measurements may expose their shortcomings. Overall, I think they do a remarkable job.

They do, indeed, and this takes fortitude and a strong sense of self-worth on the part of the reviewers.

I really appreciate learning of manufacturing foibles and greatly appreciate their inclusion in the review.

To echo Dave: great thread!

Quote:

Quote:
[
Feedback on the usefulness of the measurements or the reviewer's ability to hear?

Both. For example, I may measure a strong vibrational resonance at a specific point on a loudspeaker's side panel, but the reviewer didn't comment on any midrange congestion or coloration that could result from that resonance. There are two possibilities: 1) the reviewer should have heard the effect of the resonance but didn't; 2) the problem measures worse than it sounds, perhaps because the affected area is small so doesn't couple to the air to a any great extent, or perhaps because its acoustic output is canceled by opposite-polarity radiation elsewhere on the panel, or because its Q is so high that the resonance is never fully excited with Western-scale music.

John Atkinson
Editor, Stereophile

Given that there are so many variables that could affect the measurements, how do you figure out what causes the discrepancy between what's heard and what's measured?

Quote:
Given that there are so many variables that could affect the measurements, how do you figure out what causes the discrepancy between what's heard and what's measured?

Even without variables, no one can completely answer this question. Science is still studying the relationship between what we perceive and objective measurements of sensory phenomena.

Of course, there are knowns - such as frequency response.

Unfortunately we know enough about hearing and measurements that for most purposes there is no need for additional research. While I acknowledge there is more important things to research I wish there was even more being done to further our knowledge.

I hate to bring this up, but given the topic under discussion, it's almost unavoidable: If "science is still studying the relationship between what we perceive and objective measurements of sensory phenomena," then what's come down the scientific pike that's better than (reflexive cringe) double-blind testing, with all of its alleged flaws?

I'm also wondering if this area is in fact under active and intensive study in experimental psychology, which is the supraordinate field for psychoacoustic studies.

The science of psychoacoustics uses various forms of testing. It is rarely double blind however.

In psychoacoustics the researchers know specifically what is different in the various samples under examination. What is being tested is our ability to perceive. In fact, the subjects often know what their are listening for as well. An example is a basic hearing test.

it is good for folks to consider (or to come to an understanding of) that one of the most elaborate and complete experiments in cutting edge audio equipment vs the human ear, ie, psychoacoustics..is actually the tenure of John at the helm of Stereophile.

There is a tremendous amount of data and work done at the magazine, over the years. There have not been any scientists and blind testing involved, but that does not make the data and observations any less valuable.

I have seen more consistency in truthfulness at Phile than anywhere else that I know of. I often seem to get the idea that once in a while a reviewer might be a little nervous, if that is the right word, when one is about to give a glorious review only to read that the reviewer is "interested in what JA measures".

With the RME sound card is not the Sooloos a Class A, or at worst, Class B product? It seems to me that the sound differences are nothing more than comparing the sound quality of all the Class A Cd players, which do sound different on varying systems with high levels of resolving power.

I would doubt that most Sooloos customers will feel cheated by the sound quality with RME. And, if they do, you can always buy and outboard DAC if you wish. seems like a win-win to me.

Maybe most don't care, but lots of us would prefer a much better DAC for $10,000-plus.

Dave

Quote:
There is a tremendous amount of data and work done at the magazine, over the years. There have not been any scientists and blind testing involved, but that does not make the data and observations any less valuable.

Extremely true. Good Point.

Quote:
Maybe most don't care, but lots of us would prefer a much better DAC for $10,000-plus.

Dave

I second this. I saw & heard the Sooloos in action recently. Pretty snazzy-looking, if you're into that modernist look. Convenience & accessibility are great, I suppose, but not at that premium. And I don't know about you, but that computer screen is a deal-killer. The graphics represented on the screen looked kind of underwhelming & cheesy, too, made the system look a bit cheap, I thought.

Quote:
Maybe most don't care, but lots of us would prefer a much better DAC for $10,000-plus.

Not only prefer, but demand much better.

For $10k it better sound extremely good. Anything less is inexcusable.