The Puzzle of Perception

"Nothing is real, and nothing to get hung about."---John Lennon

"Reality is nothing but a collective hunch."---Lily Tomlin

The thing pictured above---a poyuit? a potrzebie?---puzzled me immensely when I first saw it in MAD magazine in the early '60s. It is, of course, a trick, the two-dimensional drawing giving conflicting clues to the poor brain trying to visualize it as a three-dimensional solid object. But I used it to open this discussion because, far from being fully understood, the more there is known about human perception, the more it seems there is to know. Sir Francis Crick, who, with James Watson, discovered the double-helical nature of the DNA molecule, referred to perception in 1979 as "science's greatest mystery." The retina of the eye does not behave like photographic film or a video camera tube or CCD; the ears are not microphones. After three decades of research into Artificial Intelligence, there is still no way to make a microphone aware that it hears, a video camera that it sees (footnote 1). In his 1991 book (footnote 2), Edmund Blair Bolles summarized this mystery as:

• "Our basic scientific ideas cannot grasp something fundamental to daily living.

• "The world that physicists describe is unlike the sensory one we perceive.

• "The world we feel is so surely out there seems to depend on an interior construction of our own."

Bolles's mention of the brain making use of "interior constructions" echoes something Meridian's Bob Stuart recently mentioned (footnote 3) and to which I referred in the introduction to my loudspeaker reviews last month. When someone listens to a recording on a stereo system, the reality is that two complex audio-bandwidth pressure waves emanate from the loudspeakers. The impression the listener gets that instruments and voices are hanging in space between and behind the loudspeakers is an illusion, the brain deciding that that must have been what would have been heard at the original event. In Bob Stuart's words, the brain creates "acoustic models" as a result of the acoustic information reaching the ears, models that are totally subjective.

You do this automatically---it's what you do when your ears pick up real sounds. It doesn't strike you as incongruous, therefore, that the illusion of the sounds and spatial aspects of a symphony orchestra, for example, can be reproduced by a pair of speakers in your own room. Yet there is no measurement that can be performed on the two channels of information to reveal that they represent one or all of the instruments of an orchestra. Routine review measurements examine changes in the voltage or pressure signals in just one of the information channels at a time, yet the defects of recording and reproduction systems affect not just one of those channels but both. And the audible effect of those defects is not heard as their direct measurable effect on the signals but as changes in the perceived character of those oh-so-fragile acoustic models.

System distortions that may be thought inconsequential compared with the total sound level can become significant when referenced to the clues, often very low in level, that allow the listener to create any particular acoustic model. Similarly, a large measurable difference may be inconsequential if it doesn't change the perception of that acoustic model.

While we can only measure physical differences, we have almost no idea of how changes in those measurements alter the subjective map created by our perceptive processes. Tony Rothman, in A Physicist on Madison Avenue (footnote 4), examined the fact that measuring the spectra of flutes does not reveal the subjective differences that can be perceived between different instruments. In fact, Mr. Rothman declares that the conventional wisdom that the timbral differences between different types of instruments lies in the spectra of their harmonics is wrong. Certainly a correlation can be drawn between the fact that their sounds are different and that their harmonic spectra recorded in an anechoic chamber are different. Rothman states, however, that "the spectra of a note on the oboe sampled at two different points in an auditorium differ more than the same note played on an oboe and a trumpet." Yet the subjective difference between "oboe-ness" and "trumpet-ness"---the identities of their acoustic models, if you will---is vast.

"The map is not the territory," we are all taught, yet when it comes to perception, the subjective map is the territory. It is all the evidence concerning the nature of reality we have to go on. (When that map no longer correlates with reality with any degree of fidelity, we are insane or hallucinating.)

To be able to form that subjective map's acoustic or visual models is not trivial, depending as it does on experience and education. A February 1985 BBC TV "Horizon" program written by Hilary Lawson (broadcast in the US as one of the "Nova" series on PBS) examined how "seeing" and "observation" are not the same. Observation requires of the observer a considerable degree of interpretation based on expectations and already-formed models and structures. The English language even distinguishes between "listening" and merely "hearing" by the presence or absence of "awareness." In Ms. Lawson's words: "What we see is governed by what we already know is there."

Footnote 1: For further reading on awareness and Artificial Intelligence, see the January 1990 issue of Scientific American, as well as the second edition of David Berlinski's Black Mischief (Harcourt Brace Jovanovich, 1988). (You also might want to dip into Douglas Hofstadter's Gödel, Escher, Bach: An Eternal Golden Braid, Vintage Books, 1979.)

Footnote 2: Edmund Blair Bolles, A Second Way of Knowing: The Riddle of Human Perception, Prentice Hall Press. This fascinating book also contains an excellent bibliography.

Footnote 3: In his interview with Robert Harley, "The Increasing Importance of the Smaller Difference," Stereophile, Vol.14 No.9, September 1991.

Footnote 4: Princeton University Press, 1991. Tony Rothman quotes flautist Robert Dick, examining the fact that the best flutes are indistinguishable to the 99th percentile: "But art takes place in the remaining 1%." Just as in high-end audio!


dalethorn's picture

We see upside down, so the brain has to turn that around, and then everything including motion that goes with it. We have 2 ears that each hear what they hear, then the brain has to interpret that as spatial information. Military exercises teach "off center" seeing at night, since when staring at something in near darkness it usually disappears. I've always assumed there's an audio "blind spot" that defeats a lot of blind testing of very subtle differences.