Quote: the third time in the last ~2-3 years that I have read an MF review of a piece of equipment that had a basic, measureable flaw - yet he raved about the equipment as if there was nothing wrong. That leaves JA in the unenviable position of having to write something that acknowledges the problem without overtly criticising MF.
To be fair to Michael, one of the problems I found didn't affect SACD playback or when he fed the Cary data via S/PDIF from his Sooloos server or USB from his computer. It only affected CD playback. The second problem - the downsampling of 176.4kHz or 192kHz data via S/PDFI - is extraordinarily difficult to detect purely by listening.
Quote: I think it's admirable that Stereophile makes these issues public by printing the apparently-unadulterated review alongside the post-review measurements.
Thank you. "Let the chips fall where they may" is a guiding principle of mine, since witnessing how much was buried under the carpet at other publications. Stereophile's reviewers don't see the measurements until after they have submitted their review text, BTW. It is a tribute to their integrity, I feel, that that doesn't prevent them from reporting what they perceive.
I have no problem being criticized. However, as with the Zanden review that similarly showed a measurable issue (that time due to a manufacturing error), the real issue is audibility.
I believe my reliability record is quite good <i>measured</i> on the basis of what's actually audible!
MF calls them as he hears them, making for an entertaining read. (Also, his entusiasm can be positively contagious. Which, even if you disagree with him, is a good thing for the hobby.)
The fact that JA keeps the content of the review seperate from the measurement section makes for even more fun as a reader - reconciling what the reviewer reports vs. what JA measures.
I appreciate MF's reviews; but JA's measurements can in one realm have more impact for me than MF's (or anyone's) review...for one reason only: If MF loves or hates a piece of kit, it doesn't mean I will agree, and my own audition is the final arbiter; but if JA finds significant flaws, it efficiently turns me off the product in question. If a company can't get a working unit to JA, no matter what the excuses, I am not interested. (Unless it's EL or Wes breaking shit..... .....then ya gotta factor that into your equation.)
______________________
However, given the cognitive and measured dissonace on one review item, I would love to see a tie breaker review for that Mu Fi Supercharger!
Did EL manage to break it like he did the Totems, or did MF swing and miss?
I'm with you. With Michael's super resolving system, if he is not hearing something bad or off, or horrible, then there is no way I could ever hear a bad measuring piece of gear in my systems.
If something sounds great in his system, I'll bet it is and the measurements can fall where they may. Measurements are only a part of it anyway, or how else could fans of SE amps be so happy.
Quote: If a company can't get a working unit to JA, no matter what the excuses, I am not interested.
It used to be my concern that we got cherry-picked samples for review, whereas the opposite appears to be the case. As I explain in the Audio Research thread, the magazine too often gets sloppy seconds for review.
Quote: However, given the cognitive and measured dissonace on one review item, I would love to see a tie breaker review for that Mu Fi Supercharger!
Not going to happen as the Supercharger 550 is long out of production.
Quote: Did EL manage to break it like he did the Totems, or did MF swing and miss?
The latter. Whereas the more powerful 750k was a sweet-sounding amp, I felt EL called it correctly on the 550k. Michael Fremer, however, I am sure would have found the 750k _too_ sweet-sounding.
Quote: Roger, I see you've gone way over your one post per year limit. Give someone else a chance, eh?
Hmm, I didn't think posting frequency correlated to the value of a post.
Anyway, thanks to everyone else for your considered replies; I feared walking into a sh1tstorm here!
Quote: I'm with you. With Michael's super resolving system, if he is not hearing something bad or off, or horrible, then there is no way I could ever hear a bad measuring piece of gear in my systems.
If something sounds great in his system, I'll bet it is and the measurements can fall where they may. Measurements are only a part of it anyway, or how else could fans of SE amps be so happy.
I guess that's my point. It appears that most readers are happy to accept that reviewers are generally always right. The implications are that (i) if a measurement shows a problem that the review doesn't observe, then the problem is not audible, and (ii) that reviewers' comments are always more relevant than measured results.
(Forgive me MF for using you as the example, purely due to what I happened to remember, but) what I am asking seems to be anathema to readers of Stereophile and like magazines, i.e. if we assume reviewers are basically infallible, how will we ever know when a reviewer has got it wrong?
There could be many reasons for a reviewer getting it wrong, e.g. gradual loss of hearing faculty, distraction during the review process, who knows. But if we assume that the reviewers' word is the final say, and if (for practical reasons) each piece of gear is only generally listened to be one person, then we won't even really know. I'm not sure of a way around this, without the impactical idea of having each piece of gear reviewed by 2 or 4 or more different reviewers, each without knowledge of the previous reviewer's comments.
And and at the risk of using the naughty words, if reviewers' subjective comments are always deemed to hold primacy over objective measurements, how are we to interpret the measurements? JA often refers to correlations of (objective) measurement results with perceived (subjective) responses. Obviously we are a long long long way from these correlations being even remotely definitive, but it seems that for most people, the measurements mean nothing if "contradicted" by the reviewer's comments - which seems unsatisfactory, especially for "large" measured problems.
...and if (for practical reasons) each piece of gear is only generally listened to be one person, then we won't even really know. I'm not sure of a way around this, without the impactical idea of having each piece of gear reviewed by 2 or 4 or more different reviewers, each without knowledge of the previous reviewer's comments.
This would be a great idea. Already practiced by other magazines, like eg the Norwegian "Fidelity".
Or go back to the wavelength tube amp...JA's measurements said "broken"( I paraphrase of course)and Mikey said "heaven"...yet how many wonderful Steve Hoffman remasters used the wavelength amps?? I for one trust both Mikeys ears AND JA's measurements....what we hear as good doesnt always mean ruler flat measurements(speakers???)and what measures perfect doesnt always mean good sound( 80's receivers!)I use both hearing and measurements...really is there any among us that wouldnt want or would turn down a Zandon or wavelength????
Quote: If something sounds great in his system, I'll bet it is and the measurements can fall where they may. Measurements are only a part of it anyway, or how else could fans of SE amps be so happy.
Jim, I think you miss my point, which is that if we blindly assume that all reviewers always hear and report accurately, you never allow for the possibility that they get it wrong (for whatever reason). To me, that seems admirably loyal, but a tad naive.
[This is true *regardless* of what the measurements corresponding to the review show, which is an argument not really able to be proved one way or the other, at least not without going to an awful lot of effort.]
To be fair to Michael, one of the problems I found didn't affect SACD playback or when he fed the Cary data via S/PDIF from his Sooloos server or USB from his computer. It only affected CD playback. The second problem - the downsampling of 176.4kHz or 192kHz data via S/PDFI - is extraordinarily difficult to detect purely by listening.
Thank you. "Let the chips fall where they may" is a guiding principle of mine, since witnessing how much was buried under the carpet at other publications. Stereophile's reviewers don't see the measurements until after they have submitted their review text, BTW. It is a tribute to their integrity, I feel, that that doesn't prevent them from reporting what they perceive.
John Atkinson
Editor, Stereophile
I have no problem being criticized. However, as with the Zanden review that similarly showed a measurable issue (that time due to a manufacturing error), the real issue is audibility.
I believe my reliability record is quite good <i>measured</i> on the basis of what's actually audible!
Actually, I don't mind these "errors."
MF calls them as he hears them, making for an entertaining read. (Also, his entusiasm can be positively contagious. Which, even if you disagree with him, is a good thing for the hobby.)
The fact that JA keeps the content of the review seperate from the measurement section makes for even more fun as a reader - reconciling what the reviewer reports vs. what JA measures.
I appreciate MF's reviews; but JA's measurements can in one realm have more impact for me than MF's (or anyone's) review...for one reason only: If MF loves or hates a piece of kit, it doesn't mean I will agree, and my own audition is the final arbiter; but if JA finds significant flaws, it efficiently turns me off the product in question. If a company can't get a working unit to JA, no matter what the excuses, I am not interested. (Unless it's EL or Wes breaking shit..... .....then ya gotta factor that into your equation.)
______________________
However, given the cognitive and measured dissonace on one review item, I would love to see a tie breaker review for that Mu Fi Supercharger!
Did EL manage to break it like he did the Totems, or did MF swing and miss?
Buddah,
I'm with you. With Michael's super resolving system, if he is not hearing something bad or off, or horrible, then there is no way I could ever hear a bad measuring piece of gear in my systems.
If something sounds great in his system, I'll bet it is and the measurements can fall where they may. Measurements are only a part of it anyway, or how else could fans of SE amps be so happy.
It used to be my concern that we got cherry-picked samples for review, whereas the opposite appears to be the case. As I explain in the Audio Research thread, the magazine too often gets sloppy seconds for review.
Not going to happen as the Supercharger 550 is long out of production.
The latter. Whereas the more powerful 750k was a sweet-sounding amp, I felt EL called it correctly on the 550k. Michael Fremer, however, I am sure would have found the 750k _too_ sweet-sounding.
John Atkinson
Editor, Stereophile
Roger, I see you've gone way over your one post per year limit. Give someone else a chance, eh?
Hmm, I didn't think posting frequency correlated to the value of a post.
Anyway, thanks to everyone else for your considered replies; I feared walking into a sh1tstorm here!
I guess that's my point. It appears that most readers are happy to accept that reviewers are generally always right. The implications are that (i) if a measurement shows a problem that the review doesn't observe, then the problem is not audible, and (ii) that reviewers' comments are always more relevant than measured results.
(Forgive me MF for using you as the example, purely due to what I happened to remember, but) what I am asking seems to be anathema to readers of Stereophile and like magazines, i.e. if we assume reviewers are basically infallible, how will we ever know when a reviewer has got it wrong?
There could be many reasons for a reviewer getting it wrong, e.g. gradual loss of hearing faculty, distraction during the review process, who knows. But if we assume that the reviewers' word is the final say, and if (for practical reasons) each piece of gear is only generally listened to be one person, then we won't even really know. I'm not sure of a way around this, without the impactical idea of having each piece of gear reviewed by 2 or 4 or more different reviewers, each without knowledge of the previous reviewer's comments.
And and at the risk of using the naughty words, if reviewers' subjective comments are always deemed to hold primacy over objective measurements, how are we to interpret the measurements? JA often refers to correlations of (objective) measurement results with perceived (subjective) responses. Obviously we are a long long long way from these correlations being even remotely definitive, but it seems that for most people, the measurements mean nothing if "contradicted" by the reviewer's comments - which seems unsatisfactory, especially for "large" measured problems.
...and if (for practical reasons) each piece of gear is only generally listened to be one person, then we won't even really know. I'm not sure of a way around this, without the impactical idea of having each piece of gear reviewed by 2 or 4 or more different reviewers, each without knowledge of the previous reviewer's comments.
This would be a great idea. Already practiced by other magazines, like eg the Norwegian "Fidelity".
Or go back to the wavelength tube amp...JA's measurements said "broken"( I paraphrase of course)and Mikey said "heaven"...yet how many wonderful Steve Hoffman remasters used the wavelength amps?? I for one trust both Mikeys ears AND JA's measurements....what we hear as good doesnt always mean ruler flat measurements(speakers???)and what measures perfect doesnt always mean good sound( 80's receivers!)I use both hearing and measurements...really is there any among us that wouldnt want or would turn down a Zandon or wavelength????
Jim, I think you miss my point, which is that if we blindly assume that all reviewers always hear and report accurately, you never allow for the possibility that they get it wrong (for whatever reason). To me, that seems admirably loyal, but a tad naive.
[This is true *regardless* of what the measurements corresponding to the review show, which is an argument not really able to be proved one way or the other, at least not without going to an awful lot of effort.]
Measurements is only part of the story, and to me what's important is what MY ears tell me, not what someone else tells me.