The AES website notes that the Richard C. Heyser Memorial Lecture series was established in May 1999 by the AES Technical Council, the Board of Governors, and the Richard Heyser Scholarship Fund to honor the extensive contribution to the Society by this outstanding man, widely known for his ability to communicate new and complex technical ideas with great clarity and patience. The Heyser Series is an endowment for lectures that will bring to AES conventions eminent individuals in audio engineering and related fields.
With thanks to the AES Technical Council for their permission, the preprint of John Atkinson's lecture is presented here.—Ed.
Richard C. Heyser Memorial Lecture: "Where Did the Negative Frequencies Go?"—John Atkinson, Editor, Stereophile magazine
"Even a cursory read of the academic literature suggests that in audio, all that matters has been investigated and ranked accordingly. But his 40-year career in music performing, record engineering and production, audio reviewing, and editing audio magazines leads John Atkinson to believe that some things might be taken too much for granted. The title of his lecture is a metaphor: all real numbers have two roots, yet we routinely discard the negative root on the grounds that it has no significance in reality. Perhaps some of things we discard as audio engineers bear further examination when it comes to the perception of music. This lecture will offer no real answers, but will perhaps allow some interesting questions to develop."
Good evening, ladies and gentlemen. It is an honor to have been invited to present this evening's lecture in memory of the late Richard Heyser. Audio theorist, engineer, reviewer, scientist at the Jet Propulsion Laboratory, inventor of Time Delay Spectrometry, and Audio Engineering Society Silver Medal recipient, Dick was a man I was privileged to have met just the once, at an AES meeting in London in March 1986. His comments that night gave me much to ponder in the years ahead. I was also in the audience for the presentation of his final two papers to the AES, given by telephone at the fall 1986 AES Convention in Los Angeles, from his hospital bed. I had not realized until that evening that his illness was terminal.
My wife got to know Dick well when they both worked for Audio magazine; she remembers going round a Consumer Electronics Show with him. Before entering each exhibitor's suite, Dick would cover up his name badge: "That way they won't know who I am," he said with his usual modesty, "and I will hear the system as it really is, not how they want 'Richard Heyser' to hear it."
It is also an honor to follow in the footsteps of such visionaries as Ray Dolby, recording engineer Phil Ramone, futurist Ray Kurzweil, mathematicians Manfred Schroeder and Stanley Lipshitz, film-sound pioneer and editor Walter Murch, Andy Moorer of Sonic Solutions and Adobe, Roger Lagadec, Kees Schouhamer Imminck (who developed the optical data-reading technology used in the CD), Karlheinz Brandenburg of MP3 fame, and acoustician Leo Beranek.
When Robert Schulein of the AES Technical Council e-mailed me last summer to invite me to give this lecture, I was sure that a mistake had been made. The gentlemen above invented the future. By contrast, I am just a storyteller; worse, I am a teller of other people's tales, including tales told by some of the people above.
"Writing about music is like dancing about architecture," Laurie Anderson was once supposed to have said, and to be an audio journalist is not too different. However, as a generalist in a world of intense specialization, I think I can dance a step sufficiently varied to cast some interesting shadows.
I am sure that some of the questions I will ask in the next 50 minutes or so have already been answered, perhaps even by one or more of the people in this room. Nothing I will say is either original or new. Much of it has been examined in articles I have written and speeches I have given over the past decades. However, it is unlikely that everything will have ever been grouped together in the same presentation before. And, of course, given the large amount of ground I will be covering, I am well aware that I am skating over crevasses of deeper understanding. So I beg forgiveness for the inevitable generalizations.
Early Days
I had a schizophrenic education. On the one hand, I was an academic overachiever in the sciences. On the other, music meant more to me than any other interest at school, and I continued playing bass guitar in bands, first while I kept my nose to the scientific grindstone at university, and later when I took a job in scientific research.
I started out working, in a government laboratory, on the development of LEDs. This is my ID card at the lab—long hair was mandatory for government workers at the end of the 1960s, of course.
One of my tasks was to grow my own junctions, using a slice of a zone-purified n-doped gallium phosphide crystal and depositing a layer of p-doped material on it with a vapor-epitaxy oven. I would then cleave the material into individual dies and make transistors from them under a stereo microscope. I would characterize the charge-carrier mobility by measuring the Hall Effect with an enormous magnet—I once stuck my hand in the magnet but felt nothing, despite all the ions in my nerves presumably pressing against one side. I later worked for a mineral-processing laboratory, where I learned to pan for gold, among other skills.
But even as I began slowly climbing the scientific ladder, music pulled even more strongly, and I resigned from the lab in July 1972 to join a band that had just been signed to Warner Bros., and was to make an album at Abbey Road Studio and then embark on a tour of America. Well, we made the album, but our manager did a runner with the advance from Warners and the LP was never released. One memory I have of Abbey Road was this young tape op who, one lunchtime when the producer and engineer were at lunch, sat at the console and did a superb mix of one of our songs.
This slide is a montage of two photos I took in Abbey Road's Studio 3—you can see that the tape op sitting by the 16-track Studer machine was a youthful Alan Parsons!
For the next four years I played with other bands, toured, and made other albums, but it eventually became clear that I would need a steadier source of income, and in September 1976 I joined the British magazine Hi-Fi News & Record Review as an editorial assistant. At a magazine devoted to audio equipment and recordings, I felt as if the scientific and musical sides of my brain could finally coalesce. And working on audio magazines is what I have done ever since.
As I said, I am a generalist in a world of specialists. The problem with being a generalist is the vast amount of information published in every field. It is impossible to stay current. Back when the Scientific Method was a radical new idea, and science was the preserve of wealthy gentleman amateurs, it was just about possible for a single person to know everything. But those days are long gone . . . one group of researchers reckon that 1.3 million articles were published in scientific journals in 2006 alone.
I am also old enough that my education in electronics and audio was exclusively based on tubes. Even the logic circuits I constructed at school used tubes! But looking back, I think there was one experience that foreshadowed my career as an audio reviewer. For one of my bachelor's degree final exams, I was handed a black box with two terminals and had to spend an afternoon determining what it was. (If I recall correctly, it was a Zener diode in series with a resistor.) That experience is echoed every day in my endeavors to characterize the performance of the audio components reviewed in Stereophile—every product, be it speaker, amplifier, CD player, is fundamentally a black box with input and output terminals. All I have to do is ask the question "What does it do?" And remember that testing a product is not just a case of pressing "F9" on the Audio Precision; you are faced with trying to get into the head of the designer and asking, Why did he do it this way? What is the trade-off the designer has felt worthwhile? (There are always trade-offs.) And why?
Concours d'elegance
I am addicted to elegant ideas. When I first realized that the square root of negative 1, i, could be visualized as meaning a rotation of 90° into a second dimension of what was hitherto a one-dimensional number line, it was a moment of satori. In the one-dimensional world of numbers, the concept of the square root of negative 1 is meaningless. But by adding a new dimension, you enter a new, rich reality where i does have meaning.
But it didn't take me long to realize that elegance is not always equivalent to truth. As a teenager, I thought that the hypothesis of the Static Universe propounded by Fred Hoyle, along with Thomas Gold and Hermann Bondi (whose passing significance I'll mention later), was supremely elegant. (And it didn't hurt that, as a science-fiction fanatic, I was familiar with Hoyle's fiction.) Hoyle's idea was that, as the universe expands, it causes new matter to be created, if I remember correctly, at the rate of one hydrogen atom per century in a volume "equal to the Empire State Building," so that if you took a series of snapshots of the universe, one every billion years, they would all be identical. Of course, as soon as the cosmic microwave background was discovered, Hoyle was proved completely wrong. (This is ironic, as Hoyle had invented the term "Big Bang Theory" to disparage what turned out to be the correct theory.) But the Big Bang Theory means that the universe had a beginning and will have an end, which strikes me as inelegant in the extreme.
I am also fascinated by things that don't seem to fit. For example, when I first studied the Periodic Table of the Elements, it struck me as very strange that water is a liquid at normal temperature and pressure. If all you knew were the properties of its constituents—two of the lightest elements in the Periodic Table—you would expect water to be a gas like hydrogen sulfide, but less dense and less smelly.
But water obstinately isn't a gas, and we all take for granted that it isn't. In fact, our lives depend on it not being a gas. It takes a deeper knowledge of the properties of water to understand why it doesn't fit.
It is the combination of elegance and apparent anomalies that I will be talking about in this lecture. Which brings me to an explanation of its title:
COMMENTS
......blind testing mentioned by a person that doesn't believe in doing them. Aren't they "too difficult" according to yourself? Wait......nevermind just don't answer that because I know it will end up with you Mr Atkinson making some EXCUSE as to why you can't do blind tests for the magazine when it comes to reviewing products. ZzzzzzzzzzZzzzzz.
Sure is a lot of blind testing mentioned by a person that doesn't believe in doing them.
A man sees what he wants to see and disregards the rest, eh? In the 15,000 words of this preprint, there are just a few paragraphs devoted to a discussion of blind testing.
Aren't they "too difficult" according to yourself?
As I say, I took part in my first blind test 35 years ago and since then have been involved in well over a 100 such tests, as listener, proctor, or organizer. My opinion on their efficacy and how difficult it is to get valid results and not false negatives - ie reporting that no difference could be heard when a small but real audible difference exists - was formed as the result of that experience.
John Atkinson
Editor, Stereophile
In other words JohnnyR, Mr Atkinson DID find excuses not to use blind testing but found them worthwhile enough to mention them in his Heyser speech funnily enough.
A. Join a band as a second rate bass player
B. Fail to get a record contract
C.Pretend he's an electrical engineer
D.Get an offer to ruin.....ermm I meant run Stereophile
E. Proceed to ruin........ermmmmm excuse me......run Stereophile like Hitler would.
F. Spend his time talking online instead of actually doing any real work.
G. Make excuss about his busy schedule and why he can't do DBTs
H Profit!!!
Johnny makes plywood boxes in his mother's basement, and then claims he is a speaker expert.
Trolls audio websites.
That is all.
JRusskie seems to have become a failed audio (and DBT) expert in one easy step...
A. Failed...
Hitler? I don't know whether to laugh or cry.
I asked a question of Harman Testing Lab in the Computer Audiophile forum, and post #11 is the answer, which I thought was perfect. i.e., let the testee control the switch, so whatever adjustment time they need they can accomodate.
http://www.computeraudiophile.com/f12-headphones-and-speakers/behind-harman-testing-lab-14112/
Other observation: The comments in this Stereophile article about the noise floating free of the recording due apparently to having a higher playback bandwith - that made my day.
On R.C. Heyser: His description of the 1971 Sylmar quake, hearing a low-frequency "tone" prior to the shaking, got him started on comparing earthquakes and woofers and low-frequency playback in rooms, and he got beat up on a lot for those writings. But having been involved with both, those observations of his were like gold in the bank. Saved me lots of money.
Aren't you the dude who "tested" the Shure SRH 940 headphones and claimed they were almost as good as the 800HDs from Sennheiser?
I own the SRH940s. I was somewhat willing to believe you, but when I saw that your source was an Ipod, I just flicked your review in the trash.
You chose a 200$ source to test 325$ and 1500$ headphones? And you believe your results mean something?
My review of the 940, agreed upon by more than 95 percent of respondents, was vetted with specific music tracks listed in the review and subsequent comments, using ipods alone, ipods with analog amps, and desktop DACs and amps. Some of the naysayers are either impatient or just grumpy and disagreeable, which the comments in this Stereophile article perfectly illustrate.
How nice of you to call people who question your methodology grumpy, disagreable, etc. I question your methodology because it makes no sense to test 325$ headphones with an ipod. It makes even less sense to test 1500$ headphones with an ipod...
I actually listened to the Shure SRH940 and HD800 today, using my ipod Nano. You are correct, they both sound almost the same. They both sound like shit!
I know from experience that these headphones really shine with a good source, and especially, proper amplification. As good as the Shure is (I own them), it is no match for the HD800 when both are hooked up to decent equipment.
Your test is like testing the acceleration of a Ferrari and a Prius in your 12' driveway. You won't get much better results from the Ferrari in such a limited distance.
As for the 95% who agree with you, what gear did they use (ipod)? Were those 95% SRH940 owners and fanboys?
Saying "they both sound like shit" when one of those is a revered $1500 headphone of great pedigree and popularity among actual audiophiles is telling of your judgement. I would suggest taking a deep breath, pick a headphone and some music tracks you like, then tune out the matrix and just enjoy your music. You'll be less grumpy that way.
Don't try and hide behind my words! They both sound like shit through an ipod! What a weasel you are. I did not say the 800HD sounded like shit... BUT IT SOUNDED LIKE SHIT THROUGH AN IPOD. You rigged your test, Dale.
Again, you use insults...
A long time ago one person on a big audio site challenged people like you to be more specific instead of throwing words like s**t around. There were just two persons who offered test tracks with which to compare those headphones, but everyone else (like you) just threw out expletives, and a couple members even made serious threats. Now I have over 30 years experience comparing headphones like the Stax and Sennheiser series, and quite a bit of time invested with the Audio Engineering Society as well as reading Stereophile magazine, so even if nobody in the world were to agree with my conclusions, I'm certainly qualified to test and evaluate these devices. I'd suggest you refrain from purely ranting with words like s**t and offer specific testable facts, or just go away.
This article may help you:
http://dalethorn.com/Headphone_Ipod_Versus_Amp.txt
BTW, and speaking of Prius and Ferrari, you may be one of the better off who don't mind paying $1500 for a headphone and then shelling out $2000 for a headphone amp. But let's pretend for a moment that you have to pinch pennies to get those luxurious goods, so when you finally have them you really appreciate what you have. In this latter case, were you to stumble across an ipod touch or iphone of recent manufacture, with the new Earpods ($29 purchased variety only) being driven by the Dirac DSP player, and had an opportunity to listen at length, it might make you ill to realize what you got for your $3500.
My headphone amp is a whopping 400$ (CI Audio VHP 2).
I do pinch pennies. The VHP 2 and the Shure 940 are mine.The Sennsheiser 800HD I borrowed from a friend.
Stop trying to cloud the issue with non relavant issues.
How about we let Stereophile test both headphones! First through an ipod, then with decent equipment!
Both headphones sound similar through an ipod.
Plug them into a receiver, they sound less like shit.
Plug them into a decent headphone amp, and they both sound MUCH BETTER. BUT the Sennsheiser pulls away and wipes the floor with the Shure.
Buy yourself a CI Audio headphone amp (the one I use), or any other decent headphone amp Dale... Before buying a 2500$ DAC to install on an ipod.
DUH...........................
The continuing saga of ChrisS being a troll and all around noncontributing poster. Classy act there ChrisS.
Mostly what I got from Mr Atkinson's article was the amount of photos of himself rather than Mr Heyser. I suppose a memorial lecture should be all about the presenter and how he sees fit to turn it into reasons why Stereophile does what it does regarding testing and the lack there of.
"Yes, what you think you are hearing might by dismissed as being imagination, but as the ghost of Professor Dumbledore says in Harry Potter and the Deathly Hallows, "Of course it's all happening in your head, Harry Potter, but why on earth should that mean that it is not real?"
Oh I see now. Stereophile's philosophy is based upon a ficticious Harry Potter character who spouts that what we hear is real.I'll be sure to let those people being treated for such voices in their heads that all is well and medication is not necessary.
"Footnote 2: For a long time, I've felt that the difference between an "objectivist" and a "subjectivist" is that the latter has had, at one time in his or her life, a mentor who could show them what to listen for. Raymond was just one of the many from whom I learned what to listen for."
Well if only that was true. Your readers tend to all be subjectivsts who aren't trained what to listen to but blindly ( I made a pun there since blind testing is not wanted at Stereophile) believe whatever they think they hear as the truth when in fact measuremnts will show that either they heard nothing related to the measurements or perhaps it's Dumbledore once more in their heads, make believe or wishful hearing.
All in all I found your presentation 90% about yourself and Stereophile and reasons why you defend subjective listening and gloss over things like cables and doing serous testing and throwing in various quotes and ambiguous stories barely related to the subect you were supposed to be talking about, unless that subject was muddling the understanding of Stereophile and telling everyone your own personal history instead of Mr Heyser's.
"Third, one well-known skeptic sitting in the audience tonight criticized my abstract a few weeks back on the grounds that I am just offering "hypotheses about stuff that might be just to stir the pot, while offering no real explanations."
Class act there Mr Atkinson, speak out and criticise someone you don't like or agree with in the audience who then has no opportunity to speak up and address the audience themselves. Very childish and immature.I already see how you do the same on here. "My way or the highway"
JohnnyR is once more sadly correct, Blind testing is not and wil not be used by Stereophile. Just because you Mr Atkinson, know how to push a few buttons to measure speakers, does not make you even close to the same league as Mr Heyser was. He furthered the cause of objectivity and made Audo magazine the best audio magizine there ever was. The Audio Critic is a very close second. Mr Aczel used to be one of the worst subjectivists around untill he admitted to being such and saw the light and spoke out against foolish subjective reviews. Testing will always be the most important part of audio magazines. Simply letting reviewers spout off about "blacker backgrounds" "lifted veils" or other such nonsense belittles both the reader and the so called reviewer not to mention the product under "test". Truth in testing is what Mr Heyser strove for not some silly review about rainbow foil, magic bowls or pebbles.
Mostly what I got from Mr Atkinson's article was the amount of photos of himself rather than Mr Heyser. I suppose a memorial lecture should be all about the presenter and how he sees fit to turn it into reasons why Stereophile does what it does regarding testing and the lack there of. I found your presentation 90% about yourself and Stereophile...throwing in various quotes and ambiguous stories barely related to the subect you were supposed to be talking about, unless that subject was muddling the understanding of Stereophile and telling everyone your own personal history instead of Mr Heyser's.
You have misunderstood the nature of the lecture, GeorgeHolland. The Richard C. Heyser Memorial Lectures are not intended to be _about_ Dick Heyser but instead are offered to honor his memory. The invitation for me to be the October 2011 lecturer clearly stated that I was free to choose any subject I felt appropriate; my triple career as a musician/audio engineer/audio editor and the conclusions I have formed as a result of my 4 decades' experience in those careers were felt by the AES Technical Council members to be an eminently suitable subject.
Class act there Mr Atkinson, speak out and criticise someone you don't like or agree with in the audience who then has no opportunity to speak up and address the audience themselves. Very childish and immature.
The skeptic in question had emailed me before the lecture to let me know he would be in the audience and would take an active part in the anticipated Q&A session. I included this mention in the preprint to give him the necessary opening. As it turned out, he didn't attend, but I saw no reason to delete the point he made.
Just because you Mr Atkinson, know how to push a few buttons to measure speakers, does not make you even close to the same league as Mr Heyser was.
I agree. As you can tell from my discussions of Dick Heyser's writing and thoughts in the preprint, I have an enormous amount of respect for what he achieved. I have never claimed to be his equal, nor would I. But again I must emphasize that the AES Heyser lectures are not intended to be _about_ Dick Heyser. That is your misunderstanding, I am afraid.
John Atkinson
Editor, Stereophile
I am not sure as to why a person would even want to respond to these posts. Some problems really are better ignored as to invoke the spirit of go away. I for one appreciated this lecture for what it is thank you.
I for one appreciated this lecture for what it is thank you.
Appreciate your comment. It is rare that a magazine writer has the opportunity to gather all his thoughts in one place and I owe a debt of thanks to the Audio Engineering Society for inviting me to do so.
John Atkinson
Editor, Stereophile
Whoa! There goes another synapse! Now they're both brain dead... I duelled extensively with JRusskie re: blind testing on the Forums and I daresay, he shot himself many times with his own arguments.
Saga? Can't wait for the movie version!
Sorry, Ariel... I'll try not to let it happen again.
...which is a valuable insight, I think, from this article that Mr. Atkinson has generously preprinted. His stories--which, though I am not a longtime reader, I presume also reflect the history of Stereophile--mirror the tensions found in the philosophy of science between "reality" and "perception", and the question of whether our observations are truly disinterested or if our minds shape how and what we see (or hear). And, really, what exactly does a blind test resolve? That the reviewer likes one piece of equipment over another, presumably without factoring in aesthetics or cost? But "liking" one thing better than another is still a subjective judgment, and who's to say YOU would share that judgment? Never mind the problem of translating sounds heard by another into words on the magazine's page and then back again into the sounds heard by the reader when he or she auditions the gear. Or the issue of whether sound quality can be adequately described by a series of measurements, any more than colour can be described strictly in terms of wavelength, or consciousness in terms of vibrating atoms in your brain. That's not to say these things aren't significant, but to say that they add up to the sum of our human experience is absurd.
Thanks for a reflective and enjoyable article (at least in my mind).
The purpose of blind testing is soley to see if you can hear a difference between two units. After all the subjective rants of "oh this lifted several veils of crud from the liquid sound" or "I heard a definite improvement" I find it sad that Sterophile keeps saying that DBT or even SBT are not a valid way to test those claims. Either you can hear a difference or you can't yet those that claim they do shy away from proving that fact with a blind test. Afaid they might be shown to be wrong is my opinion, so Stereophile allows them to go merrily along in their own self delusions while spending way too much money on sham products.
In a properly set up DBT, let's say for two amplifiers, what exactly will the listener be listening for?
(And what if the lamb heard a difference, but not the shepherd boy?)
Are you seriously so dense that you can't figure it out for yourself? Perhaps that's why you depend upon Stereophile to tell you what to buy?
http://home.provide.net/~djcarlst/abx.htm
http://www.matrixhifi.com/contenedor_ppec_eng.htm
Read and learn.
So Georgie,
If the lamb can hear that Amp A can make the test system sing in a range of 10hz-60khz, but the shepherd boy can't hear in that range so can't tell the difference between Amp A and Amp B, which amp should the shepherd boy buy?
No you may address me as George or Mr Holland. Georie is an attempt at making fun but then again that's about all you know to do anyway.
Please ask a relevant question or just shut it.
The lamb would first have to prove they can hear a difference with a dbt
The shepard can buy whatever he wants.
The big bad wolf tried to convince both that the expensive boutique amp was the one to buy
They told him to shove it and bought a less expensive but well built amp and they all lived happily ever after except for Mr Big Bad who soon folded his shop due to no sales.
So Georgie,
Let's make this question relevant....
Let's say we're testing two amplifiers with two listeners. The first listener is an 18 year old young lady who is trained in classical piano at a Grade 10 music conservatory level. She can hear that Amp A has an excellent range 16hz-40khz through the test system, but even though Amp B doesn't have the same range, she likes the "sound" of it better. The second listener is the shepherd boy who's grown up now. He's 54 years old, likes big band jazz, but his hearing has been damaged by working with heavy machinery without hearing protection. He can't hear a difference between the two amps.
Which amplifier should the ex-shepherd buy?
The 18 year old can hear to 40KHz? Were her parents bats?
"Like" doesn't have anything to do with blind testing. You don't pick which one you "like" you see if you can tell WHICH amp is playing. You don't even know how a dbt test is run I can see already.
Who cares which amp they buy? Maybe the people selling them do but that is completly irrevelant to dbt. You have no clue as to what blind testing is all about.
Georgie,
Are you a real person or a computer generated figment from JRusskie's russian clone of an old IBM PC? Do you know anyone with normal hearing? Do you know how real people shop or do you isolate yourself in the sanctuary of your closed mind and order everything on-line after reading extensive reviews in Consumer Reports?
Oh so you are an expert on double or single blind studies. The ABX system is a proven dbt way to do things. Just because the results have you so upset, you claim the people doing the testing are doing it wrong? Laughable. Tell me some more jokes.
...Swinging down the street so fancy-free..."
In fact, Georgie, my major(s) for my undergraduate degree were in Developmental Psychology (including Perception) and Statistics (including Research Methodology). So yes, the set up and methodology shown in those links are crap and the results are laughable...
Get thee to a local college and enroll in a first year research methodology course. Have fun learning!
Either address me as George or STFU you stupid little boy. I think you majored in being a twit and smart ass. Who can take anything you say as serious? Grow the fuck up already. You act out like a lil boy with the IQ of a rock.
Go tell the people who make the ABX test system what you just said and see how they laugh you out of the room. You bring nothing to this discussion other than what you don't agree with , with zero facts to back up your claims. Come on show us all how the ABX test methods aren't any good or why the blind testing done in the othe link was faulty. Better yet tell Harman Kardon that their blind testing techniques are faulty and not worth doing.
Georgie,
You must be running out of neurons if you don't trust your own eyes and ears. Yes, the facts are out there.
.is what SBT and DBT are all about. Using your eyes to test audio products? Well yes I can SEE that YOU would have to look so you would know which one is "better".
I find it sad that Stereophile keeps saying that DBT or even SBT are not a valid way to test those claims.
Please do not put words in my mouth. That is not what I have said. What I _have_ written is that to design a blind test that limits the variables to just that which you are interrsted in and that produces valid results when there is a small but real audible difference is complicated and time-consuming. The literature is full of poorly designed and performed blind tests that have been proclaimed by audio skeptics as "proving" that there are no audible differences. Such people demonstrate both their ignorance of the Scientfiic Method and their unquesitoning faith in "Scientism."
John Atkinson
Editor, Stereophile
JohnnyR was right more EXCUSES.
It's pretty simple Mr Atkinson but then having the will or gumption to put a dbt into the line of testing is the first thing you have to have,
You obviously don't have that or just don't care so it's a moot point anyways.
.nor ever will when money is involved.
I would hate to see magazines and websites like Stereophile become intimidated by naysayers who demand "proof" of everything they say, in advance or after the fact. There's a lot of that in mainstream media, due no doubt to controversial topics and false information being fed to reporters. But come on, people - this isn't a mainstream news outfit reporting life or death stories. We have here an incredibly rich article full of facts that can be researched and questioned with references that are well established over time. Instead we have people questioning the author's motives or his pursuit of truth? I think people who are looking for "The truth" should be looking in a religious forum, not a hi-fi forum. There's very little you can "prove" on these topics - the value here is the very informed opinion that costs you nothing.
I think you have it all backwards there friend.....
"I think people who are looking for "The truth" should be looking in a religious forum, not a hi-fi forum. There's very little you can "prove" on these topics - the value here is the very informed opinion that costs you nothing."
Religion is based upon belief and subjectivists cling to their belief, they don't go looking for proof or the scientific method. No you can't prove very much when the reviewers use subjective say so instead of actually measuring the units. Opinons are a dime a dozen or even less than that and worth next to nothing. Just look at all the opinions here.
No, George, you don't get it. You're still stuck in religion, looking for proof of something. Here you get 'information' only, and if you want 'proof' of something, you have to do the work in proving it to yourself. What *you* believe, outside of yourself, is purely opinion. Perhaps all these people looking for truth or proof are just lazy, and trying to intimidate others into doing the work for them. Like bullies.
......the facts there pal. Your "arguement" is FLAT. The only "info" Stereophile shows us is what the WANT to show us. Cables, power cords, magic bowls are off the list of even testing them in anyway what-so-ever. The reason? Ohhhhh we really don't know how to test those duhhhhh. How lame an EXCUSE is that? Stereophile is SUPPOSED to be a magazine for information NOT excuses.
"Perhaps all these people looking for truth or proof are just lazy, and trying to intimidate others into doing the work for them. Like bullies."
BWAHAHAHAHAH!!! that has to be on my "Top 10 WTF Things of 2012"
You DIDN'T just say that did you???? So let me get this straight, Stereophiles job as you see it is to just fling out "say so" and it's up to the readers to wade through the muck and mire of those reviews to try and grasp one little bit of truth? Yeah right, you must really love it then because they rarely show any truth at all.
why is it so hard to accept that double-blind listening tests are difficult to achieve as JA has explained in his lecture?
the fact that our existences are commandeered by individual perception based on thousands of variables makes it very easy for me to understand, just as how one person may enjoy spicy foods but not grapefruit or where some may hear too much bass and others not enough. so many VARIABLES!!! culture, upbringing, what sounds you are surrounded by, traffic signals, your genetic structure, your actual physical position when listening. perception is a learned skill that we do not choose to accept, it just happens and it is different for every single person.
i think THESE are the sort of differences between individuals that make DBT difficult: everyone hears differently. there is no absolute sound.
the best example of how an ear and sonic preference can change is in the study of language and sounds. the chinese language has a completely different set of sounds to that of the english language, thus their speaking intonation, laughter, and music reflect their cultural and sonic inclinations. eastern and western and andean and greek and celtic and ... and ... all use completely different scales based on their preferences of sound learned over time through language and their environments.
Thus, i often wonder do hi-fi listeners across the globe prefer different sounding systems based on their installed sonic memory? or is there a constant in terms of preference across the globe? probably not. or even more interestingly, can one find similarities in preferences in sound based on linguistic sounds of an individual region? are the frequencies accented in the german language more easily noticed by a german in his hi-fi? DBTs are a waste of time. instead of focusing why not, it is much more fun to focus on the why.
the heart of all of this lies within JA's question: where do the negative frequencies go? there are aspects to our perception of sound that simply cannot be measured because they are based on individual perception which is different for every single one of us.
If blind testing is so difficult then how did the people that I linked to manage to do so? Harman Kardon does blind testing at the drop of a hat. Go ask them how they do it so easily. Mr Atkinson's refusal to do so is simply an excuse as to not have to bring up why cables, power cords and other snake oil is indeed snake oil. He can merrily go along his way as he has for years now ignoring such products and letting his reviewers say whatever BS they want about the sham products and not have to worry one bit. He just doesn't care is the bottom line.
Georgie Porgie,
That you cite these links as authoritative sources indicates the level of your understanding of testing methodology.
......nothing to back up whatever it is you are trying to say but it is amusing.
Research methodology courses are taught in colleges and universities all over the world, even Russia...
Let us know when you and Georgie take one.
Nothing to cite other than your own wandering silly posts? Thought so.
Well little Johnny, at least Chris is A REAL psychologist. He's not a, you know, a hobbyist like yourself...
........credentials from ChrisSy just say so. Oh lets see I think I'll be a nuclear scientist now just because I say I am on the forums. There now it's a done deal. Besides how he acts out is more like a 3 year old than an adult. Some professional he is and tell us all again just what your expertise is? Trolling perhaps?
...credentials from Johnny the hobbyist speaker designer. Let's see some pictures of your Johnny brand speakers! Post some pics, or provide a link to a site where we can see these speakers of yours.
I demand to see these speakers of yours!
If JRusskie has a misguided and limited understanding of DBT, and....
Georgie has a misguided and limited understanding of DBT, then....
Are JRusskie and Georgie one and the same person?
Has anyone seen them in the same room together? Hmmmm.
Please, one of you (I guess it doesn't matter which...) ask Harman Kardon how they do their DBT's and how they use the results.
Thank you.
I haven't seen one iota of explanation from yourself yet as to why both of us and Harman Kardon and the other links George posted to are wrong. Still waiting ChrisSy.
Has HK hired you guys as DBT consultants?
Hey JRusskie,
Can you answer this one?
If A=B and C=B, then A=?
If you pass the test, then perhaps someone will hire you... But you and Georgie might have to fight over the job.
"i think THESE are the sort of differences between individuals that make DBT difficult: everyone hears differently. there is no absolute sound."
The sole purpose of DBT is to see if the person listening can distinguish between A and B. If they can't then for all practical purposes there is no difference in the sound from A and B. You and ChrisSy seem to think it's all about what the person "likes". It's a straight forward test method and "likes" has nothing to do with it.
Please explain to us all how Harman Kardon manages to use DBT all the time and do it well? I will be awaiting your reply Ariel.
So every household has a Harman Kardon product? And you and Georgie have living rooms that look like anechoic chambers? No fireplaces, of course....
Hearing a difference between a Harman Kardon product and another product in a anechoic chamber means what to you, Georgie and JRusskie?
Do you know that Ford makes the best trucks in the world?
Are you sure I didn't say 'licks". You know maybe tasting an audio product will yield just as useful results in a DBT.
JRusskie,
Now run out to your nearest Boris' Convenience store and get yourself a can each of Pepsi (do you even have Pepsi in the Former-USSR?) and Coca-Cola and set up your own Pepsi (or whatever passes for cola in Russia) Challenge.
Wiki has a nice explanation of how to do a DBT...
Once you've done your very own Peps(k)i Challenge, please send us your conclusion. We're curious...
The next step now is to get everyone in your subsidized housing project to participate in your Pepski Challenge.
Gather up that data, compare it your own conclusion and let us know how useful that information is.
I'm sure you'll enjoy the challenge of your doing your very own DBT's! (You won't even have to ask Harman Kardon to use their anechoic chamber!)
The sole purpose of DBT is to see if the person listening can distinguish between A and B. If they can't then for all practical purposes there is no difference in the sound from A and B.
And that's the problem with these tests. If a formal blind test gives results that are indistinguishable from what would be given by chance, formal statistical analysis tells us that this result does _not_ "prove" there was no difference in the stimulus being tested, only that if there _was_ a difference, it was _not_ detectable under the conditions of the test. No more general conclusion can be drawn from the results. And as I have said, it is very difficult to arrange so that those conditions don't themselves become interfering variables. Even the fact that it is a test at all can be an interfering variable, as I explain in this lecture preprint.
Please explain to us all how Harman Kardon manages to use DBT all the time and do it well?
I have visited Harman's facility in Northridge and their blind testing set-up is impressive. They have worked hard to eliminate interfering variables and their testing is time- and resource-consuming and painstaking. Even so, they have to make compromises. Blind testing of loudspeakers, for example, is almots always performed in mono. And despite the rigor of their testing, you still have anomalous results, like the Mark Levinson No.53 amplifier, which was designed with such testing but fared poorly in the Stereophile review.
While formal blind tests are prone to false negative results - not detecting a difference when one exists - sighted listening is prone to false positives, ie, it detects a difference when none exists or perhaps exaggerates the degree of difference. As neither methodology is perfect, we go with the one that is manageable with our resources. We therefore offer our opinions for readers to reject or accept in the context of their own experience and I believe Stereophile does a better job of that than any other review magazine or webzine.
If you are uncomfortable with that policy, then you should not read the magazine. And if I remember correctly, JohnnyR, you admitted in earlier discussions on this sute that you neither subscribe to Stereophile, nor do you buy the magazine on the newsstand. So why should anyone pay attention to your opinions on how the magazine conducts itself?
John Atkinson
Editor, Stereophile
So you are saying that if in a DBT the listeners could NOT tell a difference between two amps using music of their own choice then that doesn't prove the amps sound alike? Funny stuff there Atkinson. For one who thinks you should trust your ears to evaluate components you just bashed the ONE single TRUE way to test by USING YOUR OWN EARS in a DBT.
"And that's the problem with these tests. If a formal blind test gives results that are indistinguishable from what would be given by chance, formal statistical analysis tells us that this result does _not_ "prove" there was no difference in the stimulus being tested, only that if there _was_ a difference, it was _not_ detectable under the conditions of the test."
Your own words are saying that "if there was a difference it was not detectable under the conditions of the test"........oh you mean like letting the listener use the music of their own choice and switch back and forth endless times between two amps and then guess wrongly enough times so that they can't tell which one was which? LMAO if that's not proof that both amps sound alike then what sort of test WOULD prove that they do? Come on Atkinson you just don't like DBTs because they would show up so many components that people think sound "oh so better than the rest"
Opinions from you and your reviewers are the Gospel now folks. No need to test anything really just trust good ol'JA and his flunkies. Yay.
"If you are uncomfortable with that policy, then you should not read the magazine. And if I remember correctly, JohnnyR, you admitted in earlier discussions on this sute that you neither subscribe to Stereophile, nor do you buy the magazine on the newsstand. So why should anyone pay attention to your opinions on how the magazine conducts itself?"
Oh just maybe because a lot of people care for this little thing called the TRUTH? When magazines like your's take liberties with the truth by having shoddy reviews instead of in depth testing, then it's everyone's and anyone's responsibility to speak up when crappy falsehoods are published and the readers are supposed to take it all on faith. That's why. I for one do not take your opinions on anything audio related as worthwhile at all for the simple reasons that you show so much promise when you measure speakers but fail to even bother with the snakeoli products that you let slide under the radar yet let your reviewers give them glowing reviews sans any testing what so ever. Maybe that's the sighted listening bias you just spoke about yet you fail to even try with those type of products to get to the real TRUTH.
JRusskie,
If you like Harman Kardon marketing, but you're not sure if Ford makes the best trucks in the world, then get yourself an F-150 and whatever truck you used to rumble across Afghanistan with, do your DBT (just like the Pepski Challenge) and let us know what you come up with...
You are marketing TRUTH now? How pure is it?
I know some construction workers who might be interested...
JRusskie, Just looking at your response to John's post and comparing word-for-word what John wrote and what you think he says, there's such a huge world of difference!! There's a war in your head!
[Flame deleted by John Atkinson].
"Even so, they have to make compromises. Blind testing of loudspeakers, for example, is almots always performed in mono."
Well Mr Atkinson the reasoning behind testing speakers in mono is to eliminate the dreaded comb filter affect that would otherwise show up if a stereo pair were auditioned and the listener moved their head even a couple of inces. I'm surprised you didn't mention that fact but then again you think DBTs are hard to do, so if you don't know how to do them then indeed they are hard to do. *Chuckle* Any DBT done should be auditoned is such a manner. The rest of your "excuses" for not doing them is the same old same old from you, nothing surprising there.
So Georgie Porgie,
Let's say JRusskie is DBT'ing a $1500 speaker and a $500 speaker and can't hear a difference, and you are DBT'ing a $4500 speaker and a $4000 speaker and you happen to have enough working neurons to hear a difference... Which set of speakers should the ex-shepherd construction worker buy?
Even so, they have to make compromises. Blind testing of loudspeakers, for example, is almost always performed in mono.Well Mr Atkinson the reasoning behind testing speakers in mono is to eliminate the dreaded comb filter affect that would otherwise show up if a stereo pair were auditioned and the listener moved their head even a couple of [inches].
That is a consideration, of course, but in my opinion a minor one. As I had understood from Floyd Toole back in the day, the additional complexity required of Harman's physical speaker shuffling apparatus to do blind speaker testing in stereo was not justified by the results, ie, they felt that the stereo performance could be predicted from the mono results.
I don't agree with that, but more importantly, this illustrates the thesis offered in my lecture, that when you move the testing situation a step away from how the product is going to be used, you can't be sure that the assumptions you make haven't invalidated the test. As I write in the abstract to the lecture, "perhaps some of things we discard as audio engineers bear further examination when it comes to the perception of music."
BTW, I am still waiting for you to acknowledge that the criticism you made of my lecture, that it was not about Richard Heyser, was incorrect.
John Atkinson
Editor, Stereophile
More excuses? I see what Johnny meant. You are never wrong. I am pretty sure Haram Kardon knows what they are doing. Please address any criticisms to them not me.
I am afraid that comb filtering IS a big deal. That would explain why cables "sound" different. It's not the cable but the listener changing where their head is between "testing"
You will be waiting a long time for any ackowledgement about your "lecture". Stop being the primadonna already.
Georgie wrote:
"Stop being the primadonna already."
Look in the mirror and repeat those words!!!! 
So how does one differentiate speakers that sound differently, amplifiers that sound differently, pre-amps, turntables, tonearms, cartridges, DAC's, etc., if a turn of one's head makes that much difference?
Where's your reliability, Georgie? Doesn't science depend on reliability?
I can see what George is up against in here with Tweedle Dum and Tweedle Dee tag teaming and showing their ignorance.
http://www.ethanwiner.com/believe.html
Golly look what he meant.I think Ethan was banned from here ages ago for showing up Fearless Leader and his cronies and out right showing how REAL science works. BWAHAHAHAHAHAH loser boys.
http://www.audioholics.com/news/editorials/diy-loudspeakers
http://www.audioholics.com/news/editorials/diy-loudspeakers
Just bookmarking the link for future reference.
John Atkinson
Editor, Stereophile
......when you start doing a single DBT or even a SBT then you can talk about the "truth". Have you EVER designed and built your own speakers? Nahhhhhhhh you are too lazy or too "busy". Still finding plenty of time though to post online all the time though strangely enough.
Till then you aren't an engineer so take your own advice and don't comment on speaker design anymore.
If you are going to save the link then please also save this link where discussion about it unfolded.
http://forums.audioholics.com/forums/loudspeakers/83412-diy-loudspeakers-can-you-build-better-than-professional-designs.html
As you can see the original post was just one of many OPINIONS about the topic that is if you bother to read it at all. There are various OPINIONS about the topic and notice just how many of the so called "hobbyists" ended up being professional speaker builders. If you just pick and choose certain OPINIONS from the thread then you are guilty of leaving out facts.
For starters read the sixth post down by Jinjuku regarding Jeff's post that pretty much sums up where DIY has progressed.
Not real science either...
Frick and Frack strike again. Regadude and ChrisS always come up with strawman replies and ignore the links posted."Not real science "? How pompus can you get? Mr Winer measured the effects of comb filtering, what did you measure ChrisS the length of your nose when you typed that reply? You dismiss anything people link to yet show us nothing in return. Regadude, posting opinions isn't real science just so you both understand. Now run along lil boys and study real hard, maybe in another 20 years you might be able to hold your own in a discussion.
Has Winer's results been verified?
Did you know that Harman Kardon makes the best audio products in the world? And Ford makes the best trucks, right?
Or is that above your abilities like thinking?
Go ahead, put on a pink or white noise source and move your head about and tell me the sound doesn't change. You won't bother so forget it ChrisSy.
You never answer a question , you just put forth silly questions of your own. That's what people do when they don't know or are scared to try.
Moving a microphone while recording a person's voice? If that's how you do things then no wonder you don't know what Johnny was talking about. Yes the sound changes as recorded by the microphone so what? Genius.
I'll answer you this one... You and JRusskie always answer your own questions that you pose to everyone in these discussions. There's no need to provide any answer to you. As well, your attitudes and limited knowledge of the application of research methodology make civil and thoughtful discourse impossible.
So my questions to you and JRusskie are formed to reveal how each of you think whenever you provide a response.
You provide enough information for me to say that I find the "best" use of my time in these discussions is to make fun of you and JRusskie.
makes some sense of the whole darn thing.
The only duo that strikes here George, is you and Johnny. You both STRIKE OUT!
Who's doing the listening in your tests, JRusskie? Do you know any 18 year old musicians? Oh, of course not...But you probably keep company with a bunch of construction guys (lucky you!) with damaged hearing. They should all find that there's no difference between any products.
Once again, comrade JRusskie, you are on your Quixotic journey down that twisty, winding path for "truth"... Being an upstanding citizen of the former-USSR, you should know about "truth", right?
DBT is SCIENCE, and SCIENCE is ALWAYS RIGHT, especially during the heyday of the Union of Soviet Socialist Republics and you've been hanging on to this idea since the dissolution of the USSR. Hanging on to something you really want to be TRUE.
Comrade JRusskie, just because you want it, doesn't make it so.
just wanted to let you know i haven't forgotten about you.
i've been south of the equator spending time with my father and brother, but now that I'm back in the Stereophile office, i'll answer your question in full a little later.
peace out homeslice.
be nice.
Asking Johnny to be nice... Wow, you are very optimistic Ariel!
Ariel, how about doing your job and deleting post and start placing a 30 day BAN on Regadude and ChrisS for spamming this whole article day after day with nothing less than pure drivel and taunts? I've seen kids frums better moderated than this fiasco but it's already turned into a child's forum the way they act. You would think they had some dignity but we haven't seen any yet.
I believe the internet is pretty much open range, so I'm happy to just watch folks speak their minds using whatever manner to express themselves.
one thing that bothers me is when a member is unneccessarily rude to someone who is being respectful or even innocently posting unknowing of the snark they may receive. in those instances, I say something so new users are not discouraged to continue posting.
On the other hand, if you're being provocative with your initial comment, I can only expect others to react to your provocation.
******
The other policy I encourage and have stated many times is as follows
"don't feed the trolls"
******
Finally, we do delete comments from ANY member we feel is totally off-topic and insulting to an individual member, as we want the conversation to continue and bring light to the subject at hand rather than turn into a name-calling game. That being said, I'd like to delete Regadude's comment above since it just makes fun of Johnny. Also, I probably shouldn't have typed be nice. i dont know what i was thinking. I think i just wanted a rhyme in there, and also to include a general suggestion of positivity, but deleting RegaDude's comment would delete your reply, and this is a conversation I'd like to keep around.
What did I do? In response to your comment of "be nice" to Johnny, I merely joked that you were asking a lot. A lot, because Johnny is often not nice. He is that way, that is a fact. I have no responsibility for his behavior. I just pointed it out.
Don't shoot the messenger!
What is reliability?
Think, Georgie, think.
the two links provided earlier giving examples of some DBTs.
I found the matrixhifi test to be ignorable: who is the sample? how did they select these people? how are they representative of a population of listeners as a whole? in order to gather significance from these these tests, the first and most important step is determining your sample, sample size, and how you select your sample. this just seems like a bunch of friends having fun. also, since there were multiple components being switched at the same time, system synergies could have been the cause of the weaker sounding more 'hi-fi' system. maybe those components weren't right for each other, but the cheaper system just sounded better. at least in ABX, they only changed one piece at a time
what i found interesting in the ABX test was the user's ability to control the change of system component themselves. this helps eliminate the idea that the listener might feel like they are getting 'duped' or constantly searching/guessing for the difference.
Also regarding the ABX method, the # of times a difference was heard was 33. the # of times no difference was heard: 29. Interestingly, cables were the least discernable.
DBT is time consuming and for signnificant results you need a large sample size (to represent a large population of listeners). With a small sample size, as in both of these tests, you risk a greatly flawed hypothesis and will lack confidence in your results.
I don't want to whip out my textbooks, b/c i have other stuff to do, but 17 listeners is not nearly large enough of a sample size to even represent a population of 70,000 Stereophile readers (for example). Then we run into an even bigger problem of "who" is selected-->ie what type of sample you are trying to represent.
I've heard repeatedly that H/K does have a successful DBT model. I'm sure it takes them years to perform each experiment, and it is wildly expensive and time consuming. You need a large sample size for any of this stuff to matter, not a few dudes in a basement.
More excuses from Stereophile? Why am I not surprised. You scoff and blow off any attempts at doing a DBT by the people I linked to yet when it comes to "reviewing" cables and amps it's suddenly okay to accept a sighted biased review as being true? Laughable sir but then I suppose you being an emploee have to toe the line. So be it.
I give up. Go on and trust your sighted "tests" while ignoring a DBT or even a SBT. I know your boss won't let you do any. He "knows" it all. *eyeroll*
Hey George! Ariel provided you with a more than satisfactory answer. So why are you still complaining? You never heard the expression "agree to disagree"?
Ariel is right; sample size is crucial. Anyone who has a basic knowledge of statistics understands this.
George was talking about simply doing a DBT or SBT among friends like he linked to. Let's say you and 5 pals think the "Humungo" amp blows away every other amp you have listened to. So you set up a simple DBT or even a SBT to see if you can RELIABLY pick which amp is which WITHOUT sighted bias. In other words using your EARS???? I have read on the forums here how much people rely upon "What I heard" yet don't trust a simple test to prove that they can.
So lets say the "Humungo" amp is just simply "liquid, lifts 400 veils and makes the back ground blacker", what ever silly terms you wish to use, If so then you should be able to pick it out 100% of the time doing a DBT. If not and lets say you only choose right 50% of the time then that's proof you were only guessing and couldn't tell which was which. So much for saying sample size is critical. If no one can pick the "Humungo" amp from the other one then where are your rationalisations for saying it's better?
I can tell though that none of you have even bothered to try a SBT or DBT so it's all a waste of time speaking to the peanut gallery. I love how much money you both have WASTED on pricey products that only look good and sound the same. Funny as hell, go on spending YOUR money I love it
JRusskie just beat himself with his own thinking...
Johnny wote:
"I can tell though that none of you have even bothered to try a SBT or DBT so it's all a waste of time speaking to the peanut gallery. I love how much money you both have WASTED on pricey products that only look good and sound the same. Funny as hell, go on spending YOUR money I love it"
How do you know what I've bought with my money? I will go on spending my money (but not on your plywood speakers you make in your basement), that's what it's made for!
.a fool and his money are soon parted.
"...came out to play,
Georgie Porgie ran away!"
Delusions of absolute truths cloaked in "science" aren't much to hide behind, eh Georgie...
I hear our Man of La Mancha, JRusskie, is looking for his Sancho Panza to accompany him on his journey in search for TRUTH.
Happy trails!
Georgie, Look at all those scientists backing you up!
If you take a college level research methodology course, Georgie, you may not appear so laughable. At this point, there appears to be very little knowledge in what you say about reviewing audio products.
Here is how you review audio products........ looks in Stereophile and buys whatever the product of the month is. Spends too much money but doesn't care. Goes online and talks like a 5 year old and spews insults and THINKS he's smart. Case closed.
George, name ONE SINGLE product that I have bought. Just one. You and Johnny have this little fantasy in which you think you know everything and everyone.
How about you list your gear? If you are so good and knowledgeable about buying audio products, do share with us the products that are George Holland worthy. If you actually have any audio gear...
ZzzzzzzzzZZzzzzzzzzzzzzz sorry but conversing with the likes of you is boring and makes my brain hurt considering the amount of BS you spew,
Do you like what you are hearing ? If no, move on until you do. If yes, then shut up and relax. This is a hobby focused on the enjoyment of the creative output of artists. It is not about how many proverbial angels can dance on the head of a pin.
Go listen to MUSIC !
Thanks John for a very interesting and exciting presentation, lots of interesting information here and I'm surprised, to say the least, from the lack of positive feedback.
There are many who are skeptical to some of the ways of doing measurements in Stereophile and in some ways I'm one of them too, especially the way speakers are measured so close, large array speakers like the bigger Dunlavy's and some others will not sum up very nicely in this way. We do, however, not live in a perfect world and Stereophile cannot afford an anechoic chamber, so this is probably the best they can do.
I wish John would share more of this kind of information as he has gathered lots of knowledge during a long interesting career at Stereophile and other places.
Go on John :-)
Merry Christmas
Cheers harald
John, thanks for getting this lecture pre-print available for us to read. I have been looking forward to this.
I agree that there is a lot fo information combined into one lecture that anyone would need a lot of time to learn and understand the details. Pardon me for paraphrasing some of your words below.
Regarding the null result of DBTs, your description of the interpretation is in agreement with what I remember from statistics classes. I might add that a statistician would include a probability value or confidence band with the interpretation (something to the effect that 'the null hypothesis of no-difference-detctable is accepted with high probability'), and equally for the case when a difference is detected with high probability. I personally think DBTs should be done for product reviews, but agree that valid DBT's are difficult and time consuming (expensive) to do correctly, as Dr. Toole has shown in his writings.
The example of the 'backwards' impulse being not agreeable to listeners is something I have noticed in reference to digital recording. It's a wave form that does not occur naturally in music production, so reproducing it should sound 'bothersome'.
I also agree with the previous post, that more articles like this would be welcomed, to further highlight how complicated this field really is.
I have read your essay with great pleasure (all of it!) and I think it is a great example of the Liberal Arts and Science coming together. In the end, it feels like a piece of applied music philosophy, which I find fascinating. It also seems to be free of busines-oriented interests, as your opinion on cables clearly suggests. It is awesome and very unsual to meet an accomplished person who gives priority to his passions and principles over financial interests (as also expressed on your 2012 writing on the CES and Las Vegas).
I consider myself to be an audiophile that turns off the lights and tries to connect his emotions with the music with a very relaxed mind (this seems to be a category in itself, as the un-relaxed passive listeners who cannot focus on the music on a mid to long term basis tend to be very opinionated). Having said this, I recently purchased a pair of Class D mono amps that can clearly connect me to the music (Hephaestus brand). I have not ever listened to amps which are over 15k. Within similar prices, class D seems to be the better choice (but how relative this can be, Jon!)
I will continue to follow your writings with deep admiration and I thank you for making a difference on my musical experience (which is passed on to my girlfriend and my child).
Bernard
Clueless you are if you think Mr Atkinson is something special
It's a pity that some proponents of DBT as the only valid methodology have used the comments thread here to launch personal attacks against JA. Personally, I found the lecture fascinating and thought-provoking, and I thought that the nuggets of personal history provided a powerful context for the thoughtful opinions expressed. For better or worse, Stereophile doesn't restrict itself to DBTs as their only reviewing tool, but JA does measure every piece of gear his reviewers review - a practice which ensures that the opinions of the reviewers are grounded in objective data, or otherwise as the case may be. I'm not sure why this approach, coupled with a reliance on an income stream from advertising, seems to place JA in line for so much personal vitriol. If similar attacks were levelled at me in my professional life, I'd be mortified and enraged.
To cut what is risking becoming a lengthy expression of indignation short: thank-you JA for a wise and thought-provoking read.
WRONG!
"For better or worse, Stereophile doesn't restrict itself to DBTs as their only reviewing tool, but JA does measure every piece of gear his reviewers review - a practice which ensures that the opinions of the reviewers are grounded in objective data"
Do you even READ the reviews? Seriously dude. Stereophile DOESN'T do DBTs at ALL! Atkinson has said repeatedly that they are "too difficult to do". Show me ONE single DBT he has done in a review please. Plus he does NOT "measure every piece of gear his reviewers review". Cables, power cords, record demagnitizers, just to name a few. You obviously have been reading another publication NOT Stereophile.
JhonnyR
If Stereophile has the principle of not doing DBT and for you doing so is a matter of principle why oh earth would you even care to read Stereophile? Your statement above suggests you have very extensive knowledge of the content of Stereophile's articles. So you keep reading stereophile's reviews knowing they are irremediably flawed?? How perverse is this?
The real question lies on your double blind tested hatred.
Whatever your answer is I will have to assume that your hatred is a broken record and it will just go on.
He probably reads them as I do for the laughs.
......on your error filled post above so why should I comment on your assumptions? If you wish to worship Fearless Leader then by all means bow down and do so. I myself prefer to ask questions and be skeptical about his "methods" and the reason "why" he doesn't test ALL items under review.
It's a pity that some proponents of DBT as the only valid methodology have used the comments thread here to launch personal attacks against JA.
Indeed. I am not sure why I have become a lightning rod for these audio skeptics. Perhaps it is because it is a matter of religious belief on their part - having had extensive experience of blind testing, I rejected it and thus became a heretic in their eyes. Or perhaps it is just our courtesy in allowing them the space on this website to express themselves.
JA does measure every piece of gear his reviewers review - a practice which ensures that the opinions of the reviewers are grounded in objective data, or otherwise as the case may be.
As my reviewers do not see the measurements until after they have written the review, I take my hat off to them.
To cut what is risking becoming a lengthy expression of indignation short: thank-you JA for a wise and thought-provoking read.
You're welcome and my thanks to everyone else who appreciated the preprint. It was an honor to have been invited by the Audio Engineering Society to give this lecture - it's not often that you get the opportunity to look back over a 4-decade career!
A Happy New Year to everyone who surfs this site.
John Atkinson
Editor, Stereophile
You ignore the errors and just plow along blissfully gobbling up the unwarranted praise.
"JA does measure every piece of gear his reviewers review"
As I pointed out above, along with other errors on the part of Andreasmaaan,you do NOT test every piece of gear your reviewers review, but then you pretended that you do in your response. Only certain items are tested by you. Please get your own facts straight if that's even possible for you to do so.
Johnny, I wish you a great new year 2013, in which you sell 10 times more of your home made speakers than in 2012.
Since you sold zero of your speakers in 2012, you need to sell 10 X 0=0 speakers in 2013 to meet this objective. Maybe you can hire George Holland as your director of marketing and sales...
Get cracking, you have only got 364 days left to sell your quota...
[flames deleted by John Atkinson]
Unlike yourself Raghead, I will be DESIGNING and BUILDING my own speakers and learning. You do know that word "learning"? Hmmmm maybe you stopped after age 10. You on the other hand will be on here *yawning* and acting the fool and spending your money on Stereophile reviewed crapola.
So according to you Johnny, everything Stereophile reviews is crapola?! There is not one good piece of equipment that has been reviewed by Stereophile? Hundreds of reviews, not one good product?
But you, in your basement, build quality speakers...
I have challenged you before (as well as George) to name your gear. What kind of gear merits the Johnny seal of approval?
As always, you always ignore my challenge (George chickened out to). It seems you can only complain, but cannot even provide a list of your gear.
I guess ever seeing pictures of your "famous" speakers is also out of the question.
All talk and no action.
So Johnny, where are your speakers pics and specs?
In your imagination.... [flame deleted by JA]
Johnny never offered to show you anything other than his backside which I think is approriate considering the childish actions of yourself.What does it matter what our audio components are? Anything we mention would be ridiculed by yourself no doubt. Sorry but I'm not playing your silly game. Good God grow up and act like a man for once will you? 
As I pointed out above . . . you do NOT test every piece of gear your reviewers review, but then you pretended that you do in your response. Only certain items are tested by you. Please get your own facts straight if that's even possible for you to do so.
Out of morbid curiosity, I looked at the past 15 issues of Stereophile - we published 98 full reviews and follow-up reviews. Of those 98 reviews, 83 were accompanied with a full set of measurements, or 84.5%. So even if not every review includes measurements, the vast majority do, which supports Andreasmaaan's point.
I have explained to you before, in other comment threads, why we do not publish measurements of analog playback components, which is due to lack of resources. I have also explained to you why we do not publish measurements of tweaks, which is that it it is difficult to determine exactly what to measure. I don't see any reason to reopen those discussions, but that you continue to be fixated on these two issues and continue to raise them is sad.
And, as I have warned you, JohnnyR, I am continuing to delete comments from you, and others, that are nothing more than flames aimed at other readers.
John Atkinson
Editor, Stereophile
"Reviews" means just that, any item reviewed by your staff. If you think "tweaks" don't qualify then they should be labled as "Opinions" not a review. All of those full reviews you speak of were what speakers, amps, preamps, cd transposrts and the like?
Yeah I know why you don't test tweaks, that would involve you using those horrible SBT or DBT. That's a really nice system you have set up. "We don't believe in DBTs or do them or can't afford to do them"......"We don't know what to test on the tweaks"..........add the two up and you get "We don't have to show that tweaks actually do anything other than what our "reviewer" subjectivly alludes to in their sighted biased listening using their golden ears"......EXCUSES Mr Atkinson plain and simple.
Once more I ask you to read and comment about this link:
http://forums.audioholics.com/forums/loudspeakers/83412-diy-loudspeakers...
You seem to think DIY can't do anything right in your previous post in answer to Raghead. Insulting intelligent people that actually build better speakers than the status quo can produce is beyond being pompus in your case.THAT is what's SAD.
Don't you have some real work to be doing instead of being on the forums? I thought that was Ariel's job. Let him earn his pay.
JRusskie,
Look up "fixation" and "perseveration" in your dictionary. "Absolute truths" regarding audio products seems to exist only in your basement and in your head.
I looked at the past 15 issues of Stereophile - we published 98 full reviews and follow-up reviews. Of those 98 reviews, 83 were accompanied with a full set of measurements, or 84.5%. So even if not every review includes measurements, the vast majority do, which supports Andreasmaaan's point."Reviews" means just that, any item reviewed by your staff. If you think "tweaks" don't qualify then they should be labled as "Opinions" not a review.
I am not sure what you mean. You were objecting to Andreasmaan's statement that Stereophile accompanies its reviews with measurements as not being true. I offered the analysis above to show that he was correct.
Once more I ask you to read and comment about this link:http://forums.audioholics.com/forums/loudspeakers/83412-diy-loudspeakers...
You need to consult with your fellow traveler GeorgeHolland, who criticized me about linking to other sites :-)
You seem to think DIY can't do anything right in your previous post in answer to [Regadude.] Insulting intelligent people that actually build better speakers than the status quo can produce is beyond being pompus in your case.THAT is what's SAD.
With respect, you are arguing with the voices in your head here. I haven't written anything about DIY speaker designers not being able to do "anything right." What I have said is that a DIY designer, like yourself, doesn't subject his loudpeakers to the scrutiny of a disinterested marketplace, something professional designers do as a matter of course. So the question whether or not DIY designs are superior to commercial designs isn't put under independent scrutiny. It remains unsupported conjecture, as far as I am concerned.
John Atkinson
Editor, Stereophile
Mr Atkinson, you could compare DIY against manufactured speakers but of course you won't. Might upset those that can't make good speakers and I'm not talking about the DIY
Johnny plainly stated that "tweaks" are reviewed in your magazine and online yet they aren't tested, hence you don't test every component under review. You have a reading comprehension problem? How many components reviewed by Jason Serinus have been tested? uh huh, right.
So now you wuss out about commenting on links? That proves you had nothing to say in the first place about it so you shouldn't have opened your mouth.Cop out time for Mr Atkinson.
The only unsupported conjecture on here are your abilities to print honest reviews. Your ability to avoid testing alot of audio components is well known by now. Very impressive spinning you do on here.
Mr Atkinson, you could compare DIY against manufactured speakers but of course you won't. Might upset those that can't make good speakers and I'm not talking about the DIY
Back in the early 1990s, Stereophile's Corey Greenberg was one of the judges for a DIY loudpeaker competition in San Francisco. There were 2 winning designs and as part of the prize, those 2 speakers were subject to a full set of measurements in Stereophile, published in the March 1992 issue. The speakers' measured performance was good but not great and certainly didn't embarass professional designers.
Johnny plainly stated that "tweaks" are reviewed in your magazine and online yet they aren't tested, hence you don't test every component under review. You have a reading comprehension problem?
I don't believe so. JohnnyR was wrong, in that we very rarely publish full reviews of tweak products. Of the 98 reviews published in the magazine January 2012 through March 2013, there was just one review of a "tweak," Robert Deutsch on the HiFi Tuning Supreme Fuses in May 2012. ( I don't count the AudioQuest power strip reviewed by Kal Rubinson's review in December 2012 as Kal didn't make any comment on its sound quality, only on its utllity.)
How many components reviewed by Jason Serinus have been tested? uh huh, right.
The correct answer is none. But that doesn't support JohnnyR's case either, as Jason's reviews are published on the website Secrets of Home Theater and High Fidelity, which has no connection with Stereophile. I hardly believe I am obliged to support reviews of products in competing publications with measurements in Stereophile.
John Atkinson
Editor, Stereophile
"Back in the early 1990s, Stereophile's Corey Greenberg was one of the judges for a DIY loudpeaker competition in San Francisco. There were 2 winning designs and as part of the prize, those 2 speakers were subject to a full set of measurements in Stereophile, published in the March 1992 issue. The speakers' measured performance was good but not great and certainly didn't embarass professional designers."
Golly "only" 20 years ago and as we all know, nothing new has happened since then to make speaker design any better (Insert BIG sarcasm face here) Talk about a lame example Atkinson.
Yeah yeah, "full reviews" whatever, Plenty of "little revews" though all the freakin time and all they are , are OPINIONS but you know what? You let your reviewers get away with murdering the intergrity of the audio world with their stupid banter about "blacker backgrounds,lifted veils and HUGE improvements in the sound" without them having to justify their claims, Pretty slick there pal.
Oh and Jason Serinus "reviews" as posted about the recent RMAF isn't connected to Stereophile? Give us all a break, Your EXCUSES are even lamer this time than usual.
Yep typical Fearless Leader goobley gook. Did you major in Universisty in the double speak of Orwell's 1984? If not then you are gulity of plagiarizing the concept.
Golly "only" 20 years ago and as we all know, nothing new has happened since then to make speaker design any better...
Again you are arguing with the voices in your head, JohnnyR. Of course a lot has happened in the past 20 years, especially the advent of low-cost measuring equipment. But you're missing my point, which is, as I wrote earlier today, that a DIY designer doesn't subject his loudpeakers to the scrutiny of a disinterested marketplace, something professional designers do as a matter of course. You have repeatedy claimed that your speakers are as good as if not better than commercial designs. However, unlike engineers like Kevin Voecks, Paul Barton, Richard Vandersteen, Jeff Joseph, etc, your ability to pay your mortgage and feed your family doesn't depend on your skill as a speaker designer. Their's does, and it makes a difference.
Look, if you seriously believe your speaker designs are fully competitve with commercial designs, email me your measurements. I'll get back to you with my interpretation of what they mean, assuming they are as comprehensive as what I publish with every Stereophile speaker review. Put up or shut up.
Yeah yeah, "full reviews" whatever, Plenty of "little revews" though all the freakin time and all they are , are OPINIONS but you know what? You let your reviewers get away with murdering the intergrity of the audio world with their stupid banter about "blacker backgrounds,lifted veils and HUGE improvements in the sound" without them having to justify their claim...
You're not making sense. "Andreasmaaan" was discussing reviews in the magazine being accompanied by measurements. I was discussing reviews in the magazine being accompanied by measurements. You seem to be discussing something that exists only in your mind. And unlike Professor Dumbledore's quote, that doesn't mean it's real :-)
Oh and Jason Serinus "reviews" as posted about the recent RMAF isn't connected to Stereophile?
That was a show report. No-one other than yourself equates a magazine's coverage of a show with its formal equipment reviews. Are you seriously suggesting that magazines shouldn't publish show reports without including a full set of measurement data for each room they report on? Really? There isn't an audio magazine or website that does that.
John Atkinson
Editor, Stereophile
Go have another pink drink Fearless Leader.
Me let YOU interpet my frequency response? Don't make me LAUGH. You can't even interpept Wilson't ragged ass frequency response
[usual flames deleted by John Atkinson]
Look, if you seriously believe your speaker designs are fully competitve
with commercial designs, email me your measurements. I'll get back to
you with my interpretation of what they mean, assuming they are as
comprehensive as what I publish with every Stereophile speaker review.Me let YOU interpet my frequency response? Don't make me LAUGH.
You make my point for me, JohnnyR. It was a serious offer on my part. Professional loudspeaker designers send out their products for review and for the public scrutiny of possible customers. Amateur speaker designers, such as yourself, are often afraid to let anyone else judge their work.
John Atkinson
Editor, Stereophilee
I have been challenging Johnny tp provide pictures and or the specs of his speakers. He has never met my challenge.
Now the editor of Stereophile offers to look over his data. Johnny again makes up excuses to avoid providing any information on his heavenly speakers.
Big talker...
Dear Jon,
What should have been an enlightening discussion of your great lecture has been contaminated by ill-conceived comments. Some individuals have posted comments that have created an environment of nastiness.
I would seriously urge you to start deleting ill-conceived comments from now on, whether they use flamable language or not. Some people are just turned on by the flames, but others aren't. And those others would much rather focus on the joy of engaging in the music and exchanges related to it.
You do not run a state-owned site but a private one that is likely to become a lot less lucrative (in all senses) if such nasty comments are to become visible.
I am not even pointing out names and we all know who I am talking about.
I am just giving you feedback that comes from an avid reader of Stereophile online.
Finally, the correction of my use of English was mentioned by one of these individuals. Please allow me to say that English is my second language and it is oxidized. Sorry about that.
Thanks!
Thx for posting this lecture, JA!!
I also saw the Wilkinson/HTG YouTube video/interview in which you summed up some highlights of the AES lecture.
A question about the non-oversampling vs. oversampling debate ... you (JA) noted that people (subjective listeners) prefer the NOS DAC.
Was there a formal test (or survey) conducted that you may be referring to?
As far as NOS vs OS ... this has been an especially active (and debated) topic in the DIY community.
Older DACs (multibit R2R, not Sigma-Delta or one-bit MASH/bitstream) are commonly used in various DIY projects/experiments. And, all else held equal (e.g., all one does is shunt the OS chip in an older CD player), many audio hobbyists do, indeed, prefer NOS -- but many also do not (the ones who prefer OS note that once clean, well-regulated power is provided to the digital-filter IC, you get much better sound).
My own "breadboard" experiments convinces me OS is better, tho' the NOS sound has some (few!) nice qualities not present in OS. This is almost like the age-old tube vs. SS debate.
I am aware of the very $$ Zanden DAC, and that's NOS, as are the modern AudioNote CD player and the HiFiMan 602. But almost all other commercial DAC (or digital sections) equipment since the late 1980s uses digital filtering (oversampling). And, I believe, OS has won out the mass-market and high-end/audiophile market for sound, important reasons (e.g., production costs vs. fidelity; or the all-important implementation issue: it's not just the DAC chip, but, also, decoder, pwr supply, output stage, etc.)
It's difficult to isolate the benefits/drawbacks of digital-filtering -- other than the shunting/bypassing method I described above -- due to various manuf/model designs using wide array of topologies, parts, etc. In other words ... Zanden (NOS) vs. Theta or Wadia (OS).
One final note ...
The 1st- and 2nd-generation CDPs were criticized in the audiophile community for having sub-std. sound. TTBOMK, these early players were all NOS.
Then came DF chips like Philips SAA7220 (famous for the faceplate moniker: "Fourfold Oversampling Digital Filter"), and commercial (read: not modded or souped-up Philips units from Meridian/Cambridge/etc.), started to gain acceptance by serious audiophiles/reviewers.
Examples of this included various Philips/Magnavox, Yamaha and (of course) Sony models sold in department stores/mail-order-outlets at hugely-discounted prices.
So what happened to improve the sound? Was it mostly/partly OS? And/or was it better DACs? And/or better (analog) output stages? And/or better overall topologies, tighter tolerances, better PCB layout, more careful PS design?
Some of the zero-oversampling fanboyism is strange (other than the one-more-time-parasite-for-the-bored-audiophile pathology). In DIY threads and modern eBay Chinese kits (or even complete D/A NOS processors, mostly from China) new-old-stock DACs (like TDA1541 or TDA1545/1387) are used with just a simple output section and a $2 USB decoder (or S/PDIF receiver) to get data from some source.
Example:
The image below is a top-down view of a complete D/A processor based on a TDA1387 (Philips multibit R2R DAC IC from mid 1990s) -- it's a non-(zero-)oversampling config. (but uses 8 TDA1387s in parallel).
It's from China and sells for less than $125 (eBay, Taobao, etc.). I have no idea how it sounds.
The main point, however, is that: this is a fairly new offering in the marketplace ... but it's all OLD-SCHOOL ... simple topology/layout, R2R DAC, etc ... many engineers were doing this type of stuff in the mid-1980s, and henceforth.
So why is old new again?! Does NOS, indeed, sound better because most audiophiles (esp. young people, but also older geeks/reviewers who sold/got rid of their "vintage" gear) have heard nothing but oversampling or Delta-Sigma digital?
These probing queries go right to the heart of the AES lecture. Indeed, the Wilkinson video interview on Home Theater Geeks had this title: Episode 84: What Is Reality?
What is Reality?
http://twit.tv/show/home-theater-geeks/84
Another aspect of recorded/playback audio (and psychoacoustiics and related biology) is how human perception deals with:
-- NATURAL acoustic events, such as wind, leaves, birds, crickets, human voice
(IOW: random, short-duration, relatively QUIET events -- the type/duration of acoustics humans dealt with for most of the species' evolutionary history)
...vs. more "recent" acoustic exposures , such as ...
-- MUSIC/MEDIA(movies, tv/GAMING/URBAN ENVIRONMENTS
(these are much more continuous/linear, longer-duration, and louder events than what was statistically-signif. for most of hominid's evol. history).
With music, especially, there is so much more complexity (including the four std. music-theory qualities: harmony, tonality/timbre, rhythm, melody). Add words (song, opera, lyrics) .... and you have to, then, also engage the brain's symbolic/language interpreters ...hence, brain/mind acoustic organs are starting to get a real workout.
But let's not stop there: a good (= engaging) film with a complex orchestral score in the soundtrack makes things even more complicated -- i.e. the added visual-system integration! And the smell/flavor of popcorn and the sweetness/wetness/caffeine of that cola ...
I think the brain/mind can "learn" to cope with all this newer multi-tasking sensory environment. It (brain/mind) does seem to really like sensory "overload." Or movies and audio systems wouldn't be so popular.
To get to the point: Nature is a tough task-mistress. If you want to know her secrets, you've gotta have the passion for hard work (= scientific/empirical data colelction and careful analysis) AND passion for philosophical engagement. JA has done much of this in the 2011 AES lecture. Good work!
BTW: One (of the many, many) reasons classic objectivists' tests (like ABX) are flawed is that they (mostly) concentrate on short/excerpted chunks of music tracks.
You were setting up gear in the vestibule, and thus presumably moving around the area a bit. Is it not possible (indeed, highly probable) that in the short delay while they ‘demagnetized’ the LP you moved between a resonant null and resonant mode? Setting up the bass in small spaces can be challenging because moving one or two steps is often enough to produce a marked change in the resonances. We all know the vital importance of siting speakers correctly.
So yes, you heard a difference in bass. But no, this was probably the result of well-known acoustic factors and nothing to do with some kooky tweak.
| Loudspeakers Amplification Digital Sources |  |
Analog Sources Accessories Featured | |
Music Columns Retired Columns | |
Show Reports | |
Features Latest News Community | |
Resources Subscriptions |
© 2024 Stereophile
