(PDF) Stereopsis: Important Papers

Fundamentals of Stereopsis L M Wilcox, York University, Toronto, ON, Canada J M Harris, University of St. Andrews, St. Andrews, UK ã 2010 Elsevier Ltd. All rights reserved. Figure 2 shows two eyes fixating point F, so it stimu- Glossary lates the fovea, while point P is further away from the Binocular disparity – The positional differences observer. Relative binocular disparity, !, is defined as between two retinal images generated at any the angle subtended at the eyes by one point (o) minus instance by the two eyes. The brain uses this the angle subtended by the other point (y): information to extract an estimate of the depth of an ! ¼o"y ½1$ object or point. Horopter – The set of points having zero binocular This equation can be re-expressed in terms of distances, disparity that represent an arc passing through the rather than angles. Notice that fixation point. !o" iod # $ y iod Stereoacuity – The smallest binocular disparity that tan ¼ and tan ¼ ½2$ 2 2d 2 2ðd þ DdÞ can reliably be discriminated. Spatial frequency – A mathematical term describing Next, if we assume that the viewing distance, d, is large the amount of detail in a scene and any scene can be relative to the distance between F and P, Dd, then we broken down into a set of constituent sinuosoidal can assume that tan(o/2) ( (o/2), tan (y/2) ( (y/2), waveforms of different frequencies. and d ( d þ Dd, where the angles are expressed in radians. Stereoscope – Instrument invented by Wheatstone Equations [1] and [2] can then be rearranged to reduce in 1838, which uses a pair of mirrors set at right approximately to angles to focus pictures separately to the two eyes as iodDd stereograms. !( ½3$ d2 In this example, point P lies beyond the fixation point F. However, the computation of binocular disparity is the same for an object in front of the fixation point. Points in Defining Stereopsis space with images that fall on corresponding points on the two retinas have zero disparity and are said to lie along Stereopsis is a powerful cue to depth that arises as a the horopter (Figure 3). consequence of having two eyes that are laterally offset In Figure 3, points F, L, and M have zero binocular in the head. This means that each eye receives a slightly disparity and lie on the horopter, while points P and C do different image of the world, a fact one can easily confirm not lie on the horopter and therefore do have binocular by holding a finger in front of the face, and looking at it disparity. Point P lies beyond the fixation point and there- with each eye in turn. The two retinal images generated at fore its lines of sight do not cross in front of the horopter; any instant are identical in most respects, except for a it is said to have uncrossed disparity. The lines of sight for positional shift which makes it appear that the finger is point C do cross in front of the fixation point, and there- jumping back and forth as one changes the viewing eye. fore have crossed disparity. Points with zero disparity are This positional difference is illustrated in Figure 1, which fused by the visual system such that we perceive a single shows a cluster of grapes as a stereo pair (one image for image. However, points still appear fused as long as the each eye, photographs taken from each eye’s viewpoint). disparity does not exceed a certain limit, which is known Notice that the upper grape of the central group of three as Panum’s fusional area. Panum’s fusional area is illu- occludes part of the background on the left (right-eye strated in Figure 4 as the shaded zone on either side of image) that is visible on the right (left-eye image). the horopter. Even though points within the fusional area These differences in image position are known as are seen as one, their disparity creates an impression of binocular disparity, and this is the information the brain depth. Points with a disparity greater than the fusion limit uses to extract an estimate of the depth of an object or appear double or diplopic and we see that, up to a certain point, relative to where the observer is fixating. This limit, diplopic points also produce a percept of depth. disparity may also be thought of as the angular difference From the geometry outlined here, it is clear that the between a pair of points on the retinas. The geometry is critical information for stereopsis is the distance of an straightforward as illustrated in Figure 2. object relative to the fixation point, not relative to the 164 Fundamentals of Stereopsis 165 Figure 1 A stereoscopic image pair taken with two cameras. Right eye view on the left, left eye view on the right. To view, cross your eyes, until the two squares align, and focus on the middle image. Courtesy of Zeb Hodge, as posted on Flickr. P P θ ∆d F F L ω M d C iod Figure 2 The geometry of binocular disparity. The two eyes view a point P while fixating point F, at distance d from the observer. The distance between F and P is given by Dd and the interocular distance is iod. Angles o and y give the angular Figure 3 The horopter and crossed vs. uncrossed disparities. subtense of points F and P, respectively. The horopter is the set of points having zero binocular disparity, given by the dotted arc passing through the fixation point, F. In this example, L and M lie on the horopter while P (with uncrossed observer. Thus, in Figure 3, point C has a larger disparity disparity) and C (with crossed disparity) do not. than P (though of opposite sign), even though C is closer to the observer. There is an extensive literature on the it was only 150 years ago when Sir Charles Wheatstone concept of the horopter, and the coordinate systems for discovered the link between the geometry of binocular binocular stereopsis. Ian Howard and Brian Rogers pro- vision and depth perception. Wheatstone invented the vide a more complete account in their excellent review of stereoscope which uses a pair of mirrors set at right angles binocular vision and stereopsis. to present pictures separately to the two eyes (stereo- grams). When pictures of the same object or scene taken from different vantage points are viewed in the stereo- Stereograms and Binocular Viewing scope, the observer obtains a compelling view of space and volume (Figure 5). The study of binocular vision has a long history extending Wheatstone’s invention of the stereoscope inaugurated back to Euclid’s Optics, written in 300 BC, and in the the empirical study of stereoscopic vision. While consid- neglected writings of Alhazen (AD 965–1040). However, erable research has since been devoted to this topic, the 166 Fundamentals of Stereopsis study of how binocular information is extracted by the the examples used by Wheatstone in his original paper, brain, and then interpreted to provide an extremely pre- the monocular form, or structure, of the stimulus is obvi- cise estimate of relative depth, remains a fascinating puz- ous, although the direction of the disparity and depth zle. In the remainder of this article we review how relationships may not be. Even simpler line patterns stereopsis is currently assessed, when it excels and fails, (Figure 6) have often been used by scientists to investi- and describe the neural mechanisms that support it. gate the fundamental properties of human stereopsis. Classic Stereograms Random-Dot Stereograms There are two main types of stereoscopic stimuli used by In the early 1960s, while working at Bell Laboratories, Bela researchers today. What have traditionally been referred Julesz created computer-generated random-dot stereo- to as local stimuli consist of isolated single targets pre- grams (RDSs), stimuli that were to have far-reaching effects sented relative to a reference point or frame. Much like on the study of stereopsis. To generate an RDS, a pattern of random elements is produced and duplicated to provide a pair of images, one for each eye. A selected region of ele- ments is shifted in equal and opposite directions in the two P patterns. This procedure overwrites elements on one side, and leaves empty spaces on the other, in both images. The H empty spaces are then filled with samples of the random F pattern. When viewed in a stereoscope, the shifted region appears to float (or recede) in depth, by an amount propor- tional to the size of the lateral shift (Figure 7). Julesz used the RDS to highlight the problem of how the human visual system correctly matches the images in the two eyes, an issue which became known as the binocular correspondence problem. Julesz’s RDS took this problem to its extreme by providing hundreds of identical dots in the two eyes, any of which could be matched with any other dot when viewed stereoscopically. Julesz and others suggested that the brain’s strategy to solve this problem is to correlate small regions of the retinal images to identify corresponding areas. Julesz noted that the correlated area would have to scale in Figure 4 Panum’s fusional area is illustrated here in gray. size with the amount of disparity in the image (called Within this region (exaggerated here for purpose of exposition) points such as P, which lie off the horopter, H, have binocular the size-disparity correlation). While there is still some disparity but are seen as single and in depth relative to the debate as to how the size of the required correlation fixation point, F. region can be estimated prior to knowing the binocular D! D E! A! A E ι ° B c c p C! C Figure 5 Wheatstone’s diagram of his stereoscope as published in 1838. A0 and A are mirrors, which reflect images of the stereo pair (E0 , E) separately to the two eyes. The same principle is used in laboratories today. Fundamentals of Stereopsis 167 recognized immediately that since no coherent monocu- lar form is visible in either of the random-dot images prior to fusion into a cyclopean percept, his stereograms pro- vide strong proof against the Gestalt interpretation of stereopsis. To summarize, there are several important consequences in the introduction of RDSs to vision sci- ence including: 1. The robust depth percepts obtained from RDSs sug- gest that there must be a neural mechanism which is able to resolve the ambiguous matches inherent in binocularly camouflaged stimuli, without relying on Figure 6 A simple line stereogram. By cross-fusing (left image to right eye, right image to left eye) the two dark frames and the higher level interpretation of form. central dot, the relative depth of the unfilled bars become 2. The RDSs lead to the creation of simple, easy to apply, apparent. The upper bar should appear closer to the observer clinical tests of stereopsis in which monocular form or than the central dot and frame, while the lower bar should appear disparities could not be used instead of stereoscopic further away. depth. 3. The ambiguous nature of RDSs highlighted the corre- spondence problem, and the need to understand the constraints on binocular matching, which gave rise to computational models of stereopsis. The Empirical Study of Stereopsis: Thresholds and Upper Limits Stereoacuity Stereoacuity is defined as the smallest binocular disparity that can reliably be discriminated. While fixating a refer- Figure 7 A random-dot stereogram image similar to that ence stimulus, the observer indicates if a target stimulus is presented by Julez in 1964. By cross-fusing this stereo pair, one closer or further away from the fixation stimulus. Several can observe a central square region displaced in depth in front of the surrounding region. different psychophysical procedures may be used, but in all the disparity of the target is varied so the experimenter disparity, this approach remains the first step for many can assess the stereothreshold (typically defined as 75% computational models of stereopsis. The concurrent correct). Under ideal conditions, the stereoacuity thresh- introduction of the RDS and growth of the field of old tends to range from 2 to 6 s of arc, less than the width computational vision was responsible for new directions of a human hair held at an arm’s length. Ideal stimuli are in the study and computational modeling of stereopsis as of high contrast, with sharp edges, and viewed at approxi- exemplified by David Marr. mately an arm’s length; however, the following variables Although Julesz’s demonstrations prove that we solve dramatically influence stereoacuity. the correspondence problem, other work has shown that we do not appear to use each element in the display as an Contrast independent depth sample. Julie Harris and Andrew Parker It is well established that stereoacuity for luminance- measured the efficiency of stereopsis by adding binocular defined stimuli depends on stimulus contrast, or the relative disparity noise to a stereogram and comparing human intensity of dark and light regions in an image. Stereoscopic depth judgments with those of an ideal detector that cor- thresholds increase substantially at very low contrasts (near rectly extracted the disparity of each element. Human the contrast threshold for stereopsis) but this effect plateaus efficiency was very low, particularly when observers at higher contrasts where the stimuli are clearly visible. At viewed displays with jagged, rather than smooth, depth high contrasts, stereoacuity is influenced more by the profiles. This suggests that although we experience highly interocular contrast ratio than the overall binocular con- stable percepts of scenes in depth, only a small proportion trast, an effect that is also contingent on the spatial fre- of the available disparity information is actually used. quency content of the stimulus. Prior to Julesz’s introduction of the RDS, Gestalt psy- chologists believed that the percept of depth in stereo- Spatial frequency/size grams followed the internal creation of two forms, or Spatial frequency is a mathematical term describing the Gestalten, that then resulted in a percept of depth. Julesz amount of detail in a scene and, formally, any scene can be 168 Fundamentals of Stereopsis divided into a set of constituent sinuosoidal waveforms of improving with increasing duration up to a full second. As different frequencies. A high-frequency sinusoid is one we observe from the next section, this dependence on with lots of narrow stripes, whereas a low-frequency sinu- viewing time is specific to luminance-defined stimuli soid has only a few broad stripes. Much of the research on such as bars, or dots, presented at small disparities. Inves- human stereopsis in the later part of the twentieth century tigators have suggested that such performance reflects the has been based by a model of visual processing as a form of operation of a slow, sustained stereoscopic mechanism, linear systems analysis, proposed by Fergus Campbell and which responds best to small disparities. John Robson in 1968. Thus, the receptive fields of cortical neurons were often discussed in terms of their preferred spatial frequency, and it was widely held that the visual Configuration system processed information through a series of spatial An often-overlooked attribute that influences disparity frequency-tuned channels. In accordance with this, there processing is the configuration of the parts of the stereo- have been numerous psychophysical investigations of gram. Consider the work of Suzanne McKee, who showed stereoacuity as a function of the frequency of sinusoidal in 1983 that stereoacuity for a pair of thin vertical lines luminance variations. Experiments have shown that was dramatically altered by the addition of two horizontal stereoacuity improves with increasing spatial frequency lines to form a box. As shown in Figure 8 below, even the in periodic (striped) stimuli up to 2.4 cycles per degree addition of a single line connecting the vertical lines, (c/deg), at which point performance flattens. However, in results in a large threshold increase. In all cases the studies in which the stimulus size was scaled with chang- observer is asked to judge the relative depth of the two ing frequency, improvements were found up to 5 c/deg. vertical lines. In Figure 8(a), the lines are unattached and thresholds are low. The addition of a single connecting horizontal line in Figure 8(b) increases thresholds by a Modulation frequency factor of 4. The addition of two horizontal lines forming While luminance-defined spatial frequency may be im- closed Figures 8(c) and 8(d) further degrade perfor- portant for many visual functions, stereopsis has another mance. As shown in Figure 8(e), the horizontal connec- dimension, and it stands to reason that there may be detec- tors need to only imply a cohesive figure to cause a tors that are tuned to the depth modulation frequency of doubling of the threshold. a scene. For example, an egg provides a smooth change Such effects are difficult to account for based on low- in disparity and contains a low modulation frequency, level information alone. Instead, it appears that even while a leafy tree contains many regions of rapidly changing though the original disparity signal is present in the box disparity and therefore a high modulation frequency. Chris- configuration, the stereoscopic system is unable to make topher Tyler created RDS patterns that contained random use of it with the same precision because the two parts luminance variations but, when fused, appeared to modulate now belong to a single figure. There is other evidence of smoothly in depth. The frequency of the modulation could mid-level configural effects on perceived depth via stere- be manipulated, and stereoacuity measured, to provide an opsis, for instance, the perception of shape in biological estimate of a modulation frequency transfer function (MTF) motion figures influences observers’ ability to discrimi- for disparity, independent of luminance frequency. The data nate the length of body parts as they receded in depth. from such cyclopean patterns show much lower peak sensi- These phenomena show that the perceptual grouping of tivity than reported for luminance frequency, near 0.4 c/deg. parts into a whole can degrade our ability to make simple depth discrimination judgments, and demonstrate that the Duration extraction of binocular disparity is the first stage of a Stereoacuity for luminance-defined stimuli shows a complex process that extracts the depth and shape of clear dependence on exposure duration, with thresholds objects around us. (a) 4.9 ± 0.5 (b) 21.6 ± 2.5 (c) 31.1 ± 4.6 (d) 28.0 ± 4.1 (e) 12.2 ± 1.1 Figure 8 Stimuli adapted from, and thresholds (in seconds of arc) reported by Suzanne McKee in 1983. In each instance the two vertical lines are displaced in depth to assess stereoacuity and are identical in all conditions (a–e). Fundamentals of Stereopsis 169 Stereopsis at Large Disparities by Mark Edwards and colleagues, operates best when the viewing time is brief, less than 200 ms. Recent work has As described above (Figure 4), the stereoscopic system shown that for this type of stimulus, increasing the view- processes disparity information over a wide range of dis- ing time over 50 ms results in no improvement in perfor- parities at which, depending on the nature of the stimulus, mance; but results in, if anything, performance decline. the image may appear either fused or diplopic. On the horopter there is no binocular disparity between a point and the fixation location, therefore the point is perceived Similarity as equidistant with fixation. As disparity is increased, the There is evidence suggesting that depth percepts for dip- point is perceived at different locations in depth relative lopic stimuli are robust to large interocular stimulus differ- to fixation, with perceived depth increasing with increas- ences. Donald Mitchell and Gerald Westheimer used ing disparity. Within Panum’s fusional area, the disparate diplopic stereopairs containing, for example, a circle in object appears single, fused, and in depth. At some point, one eye and a cross in the other and measured depth as disparity continues to increase, fusion will be lost and discrimination. Similar resilience of stereopsis to the dis- the object will become diplopic or doubled. Surprisingly, similarity of the stimuli in the two eyes has been reported as Kenneth Ogle documented in 1952, the percept of by a number of investigators, who have shown that depth relative depth is not lost at the diplopia point. Instead, perception for large disparities is not significantly influ- there remains a compelling sense of depth over a large enced by interocular differences in contrast, or orientation. range of disparities until, for very large disparities, the now diplopic object appears to return to the fixation plane. The disparity at which the percept of depth is lost is called the upper disparity limit. Ogle categorized the The Mechanisms of Fine and Coarse Disparity high-resolution range of disparities, which provides good Processing relative depth information, as patent stereopsis; this range As is evident from the preceding sections, there are some of disparities extends beyond the fusion limit. At some substantial differences in the processing of fine- and point, as disparity is increased, the quality of the depth large-scale disparity information, particularly when the percept changes and we are no longer able to judge the large disparities are outside the fusion range. There have amount of depth, but simply have a sensation of the been a number of attempts to capture these differences direction of the depth offset. Ogle named this latter and use them to categorize stereopsis. For instance, Ogle range as qualitative stereopsis. argued that the range of disparities that can be used to make quantitative depth estimates involve patent stereop- Spatial frequency/size sis, and the binocular disparities that give rise to a more We have not included specific disparity estimates at which vague percept of depth involve qualitative stereopsis. the upper limit for stereopsis occurs, for research has Importantly, Ogle’s limits are contingent on the scale of shown that this limit (Dmax) depends critically upon the the line stimuli used to make the measurements, so they size of the stimulus, with larger Dmax obtained for wider will change as stimulus size is varied. The sustained or stimuli. Laurie Wilcox and Robert Hess demonstrated transient stereopsis distinction mentioned above appears that for Gabor stimuli (a luminance sinusoid enveloped to be based solely on the temporal properties of stimuli. by a Gaussian distribution), the size of the Gaussian en- However, the link between the transient process and large velope is the only attribute that influences Dmax; changing disparities suggests the existence of separate coarse and the center spatial frequency of the Gabors (the width of fine mechanisms that have different temporal properties. the stripes in the pattern) has no effect on the upper limit. This proposal is consistent with the series of experiments They concluded that depth percepts from diplopic targets by Wilcox and Hess describing first- and second-order are mediated by the second-order, or envelope-based stereoscopic mechanisms. The current consensus is that system, which is insensitive to the luminance-defined stereopsis has a high-resolution system that is sensitive to detail within a stimulus. luminance changes in an image. This is the classic stereo- scopic mechanism studied since Wheatstone’s invention Duration of the stereoscope. The second-order system is sensitive High-resolution stereoacuity judgments for edges and to contrast variation, and so is able to abstract over the lines improve with increasing viewing time. Quite a dif- fine detail in an image to provide a disparity signal for an ferent pattern of results is emerging for stimuli that access object as a whole. This mechanism is most similar to a more coarse stereoscopic mechanism (second order or Ogle’s qualitative stereopsis, and is likely responsible for qualitative stereopsis). In such cases, it appears that the many of the puzzling results in the literature of stereopsis stereoscopic system is more transient and, so as proposed from dissimilar stimuli. 170 Fundamentals of Stereopsis The Depth of Moving Objects Wiesel’s Nobel-prize winning work in the 1960s and Hor- ace Barlow’s studies of disparity selective neurons in 1967. The details of stereoscopic vision described up to this These investigators were the first to record responses of point have been restricted to static stereoscopic images. binocular neurons in the cat visual cortex, and laid the However, very often in the natural world objects are in foundation for subsequent work on the neural underpin- motion, and we must determine their positions in the nings of stereopsis. Subsequently, Gian Poggio and David world as they move in depth. When an object moves in Ferster showed that disparity-tuned neurons exist in cor- depth, its binocular disparity changes with respect to the tical areas V1 and V2 of the monkey and that further, they stationary background. In principle, the mechanisms that could be classified according to their response properties underlie binocular disparity processing could be used to as tuned excitatory, tuned inhibitory, and near and far. detect motion in depth, if the change in depth is then Tuned neurons have response properties consistent with monitored across time. But there is another potential high-resolution depth judgments, while the near and far source of binocular visual information. When a point cells provide a more coarse disparity signal, and respond moves directly toward an observer, the retinal image of over a larger range of disparities. The link between neural that point moves over the left retina in a leftwards direc- disparity detectors, and the percept of depth was firmly tion and moves rightwards on the right retina. If the visual established when Blake and Hirsch conducted experi- systems were to first use monocular motion-sensitive ments in which cats were raised with alternating monocu- mechanisms to detect the motion, and then use a binocu- lar deprivation. They showed that: lar mechanism to compare those motion signals, motion in depth could be detected without the explicit use of dis- 1. Binocularly deprived cats, while visually normal on parity detectors. Using a dynamic random-dot stereogram monocular tasks, had no stereopsis. This makes the (DRDS) that contains no consistent monocular motion link between perception and the cortical effects of information, Julesz showed that the former information, deprivation. from changing binocular disparity, supports the percep- 2. Animals that viewed the world monocularly alone did tion of motion in depth. More recently, visual stimuli have not develop binocular neurons; hence simultaneous been designed to isolate the motion information. In a binocular input is critical to the normal development time-correlated random-dot stereogram (TCRDS), each of binocular neurons. eye views consistent leftwards or rightwards motion but, Based on such deprivation experiments, it is clear that point for point, there is no consistent binocular disparity disparity detectors are the basis for perception of depth information. As discussed by Harris and colleagues in through stereopsis. Electrophysiological studies from the their recent review of this topic, currently there is much late 1970s onward have revealed much about the neural debate in the literature about which source of information coding of stereopsis. For instance, Bruce Cumming and is used, and when. Motion in depth does appear to be Andrew Parker have demonstrated that neurons in pri- perceived in a TCRDS, at least for some observers, yet mary visual cortex respond to anticorrelated RDS pat- thresholds for detecting the minimum amount of motion terns, that is, RDSs with a zero-disparity region defined in depth are not improved by the addition of consistent by identical dots in the two eyes surrounding a region of motions to a DRDS stimulus. Further, there is a correla- elements that have the opposite contrast (black vs. white) tion between static stereoacuity and acuity for motion in in the two eyes. Human observers do not see consistent depth, suggesting a crucial role for disparity detectors. depth in the anticorrelated region, but some V1 neurons A very small number of studies have demonstrated that respond reliably to these stimuli. This suggests that the some observers, who cannot see changing disparity in a primary visual cortex codes disparity but this signal is not DRDS (some with misaligned eyes, and some with no sufficient to support our perception of depth. However, history of ocular problems), can instead use the motion cells in the V2 area are specifically sensitive to correlated information provided by a TCRDS. This suggests that RDSs. Recent electrophysiological experiments strongly there may be two separable binocular mechanisms for indicate that extrastriate cortical areas (beyond V1) re- detecting motion in depth: one that relies on changing spond to different patterns of binocular disparity. For disparity and another that uses monocular motion signals. example, investigators have shown that disparity tuning It appears that, when present, the changing disparity is a dominant attribute of medial temporal (MT) neurons, information is the default signal used by most observers. which traditionally have been cast as motion-sensitive units. In a recent study, Takanori Uka and Greg DeAngelis The Neural Basis of Stereopsis used electrical stimulation of neurons in monkey MT region to bias their near or far response to a stereoscopic Nonhuman Animal Studies stimulus. Note that, this result was obtained for coarse Our understanding of the neural basis of stereopsis has disparity-tuned neurons, but not for fine. In the inferotem- grown exponentially since David Hubel and Torsten poral cortex (IT), which is well known for its role in coding Fundamentals of Stereopsis 171 faces and complex forms, researchers have identified neu- cues, and how the combination process is reflected in the rons that respond to the three-dimensional (3D) shape (i.e., variance of the scanning signal. convex vs. concave) of disparity-defined surfaces. There is ongoing debate regarding the implications of the wide- spread presence of disparity selectivity throughout cerebral Conclusion cortex. One possibility is that disparity processing occurs for different purposes along separate pathways in the visual Since Wheatstone’s introduction of the stereoscope in 1838, system. Along the dorsal pathway, which includes MT, we have all enjoyed stereograms of one form or another. there may be specialization for coarse disparity processing, From the impressive large-format three-dimensional (3D) which is useful for the guidance of vergence eye move- movies shown in IMAX theaters to the Magic Eye books, ments. Along the ventral pathway, which involves V2 and there is a quality of magic in the experience of depth from IT, higher-resolution disparity signals are combined to stereopsis. Scientists too have that sense of wonder that the provide precise estimates of shape and location. This pro- simple physical fact of having two simultaneous views of posal is the focus of much of the ongoing electrophysiolog- the world results in such a vivid sense of the space between ical research on stereopsis. and taken up by objects in the world. Our goal is to under- stand how the brain achieves this feat. Although we are making considerable progress, several questions remain Brain Imaging in Humans unanswered, including: What types of neural pathways are responsible for stereopsis? How is stereoscopic information Human functional magnetic resonance imaging (fMRI) integrated with other cues to depth? How does the stereo- experiments have mirrored electrophysiological studies scopic system represent extended surfaces in depth? in showing that stereoscopic stimuli activate most, if not all, of the visual areas of the cerebral cortex. The impor- See also: Amblyopia; Binocular Vergence Eye Move- tant factors in isolating specific regions are, not surpris- ments and the Near Response. ingly, the nature of the stimulus (e.g., correlated vs. anticorrelated RDS patterns) and the task. For example, in the first systematic fMRI study of the cortical areas involved in human stereopsis, Benjamin Backus and col- Further Reading leagues found elevated activity in area V3A when obser- vers were asked to make depth discrimination judgments, Arditi, A. (1986). Binocular vision. In: Boff, K., Kaufman, L., and instead of passively viewing stereoscopic stimuli. In the Thomas, J. (eds.) Handbook of Perception and Human Performance. New York: Wiley. past 10 years, fMRI research on human stereopsis has Howard, I. P. and Rogers, B. J. (2002). Seeing in Depth, Vol. 2. Toronto, flourished, and for the most part appears to map well ON: I. Porteous Press. onto the electrophysiological data. The field of fMRI is Julesz, B. (1971). Foundations of Cyclopean Perception (reprinted in 2006). Boston, MA: MIT Press. advancing, and investigations of stereopsis are moving Marr, D. (1982). Vision. New York: W.H. Freeman. from identification of where it occurs to how it occurs. Regan, D. M. (ed.) (1991). Binocular Vision (Vision and Visual For instance, Andrew Welchman and colleagues recen- Dysfunction) Vol. 9. Boca Raton, FL: CRC Press. Sacks, O. (2006). Stereo Sue. The New Yorker June 19: 64–73. tly used fMRI to examine the visual areas in which Von Noorden, G. and Campos, E. (2002). Binocular Vision and Ocular stereoscopic information is combined with other depth Motility. St. Louis, MO: Mosby. Vision Research xxx (2012) xxx–xxx Contents lists available at SciVerse ScienceDirect Vision Research journal homepage: www.elsevier.com/locate/visres Clinical evaluation of stereopsis Gerald Westheimer ⇑ Division of Neurobiology, University of California, Berkeley, CA 94720-3200, United States a r t i c l e i n f o a b s t r a c t Article history: Principles of the design and administration of clinical stereopsis tests are outlined. Once the presence of Received 14 September 2012 the distinct sense of the third dimension by binocular vision alone and without help from monocular cues Received in revised form 10 October 2012 has been established in a patient, the examination can proceed to the measurement of stereoscopic acu- Available online xxxx ity. Best results are obtained with high-contrast, sharp, well-articulated and uncrowded elements from easily-recognized target sets, displayed with no time constraints. Polarization is the preferred method Keywords: of right/left eye separation; time-sharing at a minimum of 60 Hz on computer displays with counter- Binocular vision phase occluding goggles is a feasible procedure. Random-dot stereograms are problematic because not Stereoacuity Depth perception all observers can disentangle the coherent global disparity on a first view. Perceptual learning ! 2012 Elsevier Ltd. All rights reserved. 1. Introduction Dc ¼ ða=z2 ÞDz ð1Þ Disparity is defined in an observer’s object space and, as is evi- Forward placing of the two eyes during vertebrate evolution re- dent from the equation, depends in each instance on a, the obser- sulted in overlapping visual fields of the two eyes. The consequent ver’s interocular distance ($65 mm), on z, the target distance, and dual imaging of the same objects on the right and left retinas led to on Dz, the distance difference. It is an angle, and when a, z and Dz the development of special circuitry that ensures a unified repre- are in the same units, say cm, it is in radians. For conversion, it is sentation of the world while at the same time allowing information handy to remember that each radian contains 57.3", 3438 min or about the third spatial dimension to be extracted by comparison of 206,265 arcsec. the somewhat differing aspects of targets that arise when imaged from two separate vantage points. This is the faculty of stereopsis, a facility to gauge spatial rela- 1.1. Subjective ‘‘depth’’ versus objective ‘‘disparity’’ tionships in the third visual dimension. It is subserved by dedi- cated neural circuits grafted on the more elemental ones for It is conceptually important to distinguish between observers’ processing the object space projected by the eye’s optics on the sensory experiences as reported by them and the geometrical two dimensional retinas and from there by retinotopic relays into arrangement of the physical stimuli which can be objectively the visual brain. measured. It is helpful to maintain this separation also semantically, The geometry of the situation can be simplified to the case of a and to refer to the former as ‘‘depth’’ and the latter as ‘‘disparity,’’ point target in the mid-sagittal plane at a distance z from an obser- much as one differentiates ‘‘brightness,’’ the subjective attribute, ver with inter-ocular separation a. To a satisfactory first approxi- from the stimulus ‘‘luminance,’’ specified by physical measurement. mation when z is large compared to a, the z co-ordinate of the point can be defined by c, its binocular parallax, where c = a/z in 1.2. Stereopsis versus monocular depth clues radians. A patient’s ability to estimate c depends on a variety of factors, but this is not the subject of the current contribution, At the outset the categorical distinction needs to me made be- which is rather the judgment of differences in the antero-posterior tween stereopsis and the ability to judge the three-dimensional distances of objects. This is achieved by gauging differences in bin- disposition of objects in the visual field from other cues. With a re- ocular parallax, called disparity. When Dz is small (Fig. 1), it is re- fined perceptual apparatus and experience, it is possible to navi- lated to Dc by the equation gate exceedingly well in the visual world by what are called monocular depth cues because they are available to a patient when using only one eye. Here are some examples of monocular depth ⇑ Address: 144 Life Sciences Addition, University of California, Berkeley, CA cues: A known object subtends a smaller visual angle the more dis- 94720-3200, United States. Fax: +1 510 643 6791. tant it is, contours known to be parallel, such as streets or railroad E-mail address:

tracks, converge according to laws of perspective, nearer targets 0042-6989/$ - see front matter ! 2012 Elsevier Ltd. All rights reserved. http://dx.doi.org/10.1016/j.visres.2012.10.005 Please cite this article in press as: Westheimer, G. Clinical evaluation of stereopsis. Vision Research (2012), http://dx.doi.org/10.1016/j.visres.2012.10.005 2 G. Westheimer / Vision Research xxx (2012) xxx–xxx below the detection threshold for that variable.] As will be seen be- low, the conditions all serve to optimize performance. The disparity threshold, small in terms of angle subtended at the eye, constitutes a challenge in implementing stereo tests. It is here accommodated by the very long observation distance. In the equation, Dz and z have an inverse square relationship so that a tenfold reduction in the target distance, say from 600 to 60 cm, brings about a hundredfold decrease in the just discriminable dis- tance interval to 0.3 mm or about 1/100 of an inch. And indeed a good observer has no difficulty detecting, by stereoscopy, the indentations within the profiled head on a coin at arm’s length. While it is good practice to use objects with real three- dimensional features, the small distances in physical space when the tests are carried out in confined spaces create difficulties that, as a consequence, lead to the adoption of an altogether different strategy for stereo testing: stereograms. Instead of physically arranging test targets in the patient’s three-dimensional space of objects, a pair of two-dimensional reproductions is generated of Fig. 1. Schematic geometry of the Howard–Dohlman stereoacuity test. Peg A is the view of that space from the vantage point of the patient’s right fixed at a distance z from the observer, whose left (L) and right (R) eyes are a distance a apart. The observer’s task is to set peg B so that it is just discriminably and left eyes. These are then presented separately and each directed nearer than A. The binocular parallax of A, in radians, is a/z. With respect to A, B has to its intended eye. In this way, small front-to-back position differ- disparity Dc = a/z2)Dz. ences in three-dimensional object space are represented as small right–left positional differences in the stereogram pair. The geome- try of this conversion has been treated elsewhere under the term interpose themselves and therefore partially obscure more distant stereoscopic depth rendition, but as a guide, a 20 arcsec disparity, ones, shadows are assumed to arise from a sun shining from above. shown at 40 cm to an observer with 6.5 cm interocular distance, The fact that good three-dimensional information can be gleaned would be represented by a lateral position displacement between from purely monocular viewing, as has been the practice in visual the right and left stereograms of less than a tenth of a millimeter. arts and displays for nearly a millennium and is embodied in the Because real-space simulation of three-dimensional configura- entertainment industry so much that 3D showing is regarded as tions by controlled generation of appropriate electro-magnetic dis- an extra-ordinary event, does not mitigate the distinct, non- turbances for direct unmediated view by the observers’ eyes substitutable role of stereopsis in the every day visual experience (hologram) is still in the future, clinical testing of stereopsis now- of a patient and the impoverishment that results when absent. adays centers largely on utilizing devices that allow uncomplicated Nor does the occasional report of competent one-eyed pilots. view of suitable stereograms. These monocular depth cues, as well as the relative motion of The practical questions, apart from creating patterns with such images with head movement, highlight a problems associated with minute texture, is their display. In the early days of stereoscopy, clinical stereopsis tests. Because the aim is to ascertain, qualita- this was achieved by mirrors or prisms which inevitably require tively and quantitatively, the functioning of a patient’s apparatus care in head and eye placement. This is still the case with proce- for binocular disparity processing, special precautions need always dures in which right and left stereograms are physically inter- be taken to ensure that a patient’s response is based on detection of leaved in narrow vertical strips and optical means are used to disparity and is not secondary to judgments about target location diverge the paths by the several centimeters needed to project in 3-space that could have been made with just one eye. Many clin- them into the two eyes. ical stereo tests, therefore, include a simple check that both eyes For these reasons, the most popular way of displaying stereo- are in fact participating. grams is to show the right and left eyes’ views not side by side but superimposed. The best known example is to print, superim- 2. The paradigmatic stereo test posed on a single panel, one eye’s target in red ink and the other’s in blue–green, with non-overlapping wavelength bands, to be Consideration of one of the first and still one of the best clinical viewed through colored filters that ensure that each retina receives procedures, the Howard–Dohlman two-rod tests (Howard, 1919), only the image intended for it. They are called anaglyphs. Techni- is instructive. cally more complicated but visually less intrusive is the process It is typically implemented (Fig. 1) by showing the observer two of separation by transilluminated polarized panels, with orthogo- thin rods at a distance of 6 m, seen in an otherwise empty field nal viewers for the two eyes. In either case, viewing is through gog- against a uniformly-lit background. One rod is fixed and the other gles. Because the printed colors depend on the kind of illumination can be moved back and forth by the observer, who is instructed to and may not always be matched to the transmission of the goggles set it to appear just detectably nearer than the fixed rod. When that and hence may introduce significant interocular differences in light has been accomplished, we have the values for the three variables level, polaroids are preferred. to be inserted on the right-hand side of the equation. For example, For the future, the most promising of the techniques, and one on if the inter-ocular distance is 6.5 cm, the fixed rod distance 600 cm the verge of widespread realization, is right/left time-sharing, and the just-discriminable difference 3 cm, these values yields a made possible by computer display refresh rates so fast that the in- disparity threshold of 3 ! 6.5/6002 radians or 11 arcsec. ter-ocular delay is negligible in practice. The right and left eyes’ In this testing procedure the observation time is not limited, views are written sequentially on alternate pages and their display targets are simple, single, do not have to compete with or be dis- synchronized with a viewing device with right/left eye occlusion in ambiguated from other features in the visual field, and their visual counterphase, or by transmission through panels with rotating attributes other than disparity remain invariant throughout the circular polarization; here the analyzers in front of the two eyes process of measurement. [The visual angle subtended by the rod’s can be passive. Rapid progress in optical technology bids fair to width does change with z position, but the 0.5% difference remains advance these procedures further. The fine grain needed in stereo Please cite this article in press as: Westheimer, G. Clinical evaluation of stereopsis. Vision Research (2012), http://dx.doi.org/10.1016/j.visres.2012.10.005 G. Westheimer / Vision Research xxx (2012) xxx–xxx 3 displays will remain a hurdle. Computer monitor pixel size does Here is a list of stimulus variables that should be optimized in a not yet reach down to good observers’ stereo threshold at arm’s determination of stereo-acuity: length. Hence sub-pixel resolution procedures, known to be em- ployed by the human visual system, will have to be employed. 1. Brightness. The targets, or the background against which they are viewed, should be well in the range of photopic luminances, preferably at least 30 cd/m2 (Mueller & Lloyd, 1948; Westheimer 3. Clinical questions & Pettet, 1990). The phenomenon first described by Wilcox (1932) of a decrement of performance at very high luminances Of interest in the clinic are the two extremes of stereopsis per- (>1000 cd/m2) may also apply to strereoacuity. formance, at one end its very presence in a qualitative way and, at 2. Contrast. Stereoacuity suffers from reduced contrast more than the other, stereoacuity thresholds. To reach the latter, all interact- other hyperacuities. A Michelson contrast of a minimum of 0.1, ing variables need to be set at their optimum. But when integration but preferably 0.3 should be provided (Westheimer & Pettet, of the two uniocular pathways may have been compromised by 1990). developmental or pathological interference, a clinician wants first 3. Image sharpness. The tendency towards esoteric patterns such of all to settle the question of the presence, in its most rudimentary as Gabor patches should be resisted. Any defocusing, image form, of a subjective experience of three dimensionality that per- degradation or spatial filtering is detrimental to stereoacuity force can be attributed to the purely binocular nature of a stimulus (Westheimer & McKee, 1980b) which is best with crisp targets. and not to any of the many secondary clues that could be em- Binocular image differences, as in uncorrected or induced aniso- ployed by the one-eyed. metropia or aniseikonia, are invariably disadvantageous. Is stereopsis present? A target pair shown to the right and left 4. Exposure duration. Ogle and Weil (1958) found stereoacuity to foveas might be seen single, and even when both eyes are partici- improve by a factor of 4 as exposure duration was lengthened pating and a check test rules out suppression, the patient may still from 10 to 1000 ms. The data of Westheimer and Pettet be stereoblind. (1990) suggest that stereoacuity stimuli should last a minimum The simplest procedure is to have a patient hold a pair of knit- of 200–400 ms. ting needles one in each hand and try to have the two points touch. 5. Binocular synchrony. The comparison process that is involved in In the presence of functioning stereopsis, this can be done with an disparity detection can operate only within a binocular syn- error of less than a millimeter in the antero-posterior dimension, chrony window of a very few tens of milliseconds (Westheimer, whereas with only monocular clues the error is in terms of centi- 1979); right/left target alternation should be at a rate no less meters. Simpler still is for the examiner to extend a finger from than 30 Hz (Wist & Gogel, 1966). each hand and determine how good the patient is in estimating 6. Feature isolation. There is ample evidence that in order to dis- front–back juxtaposition. criminate disparity, features should be articulated well, sepa- Coarse and fine stereopsis. In what follows most of the emphasis rated by a minimum of 10 arcmin in the fovea (Westheimer & is on the measurement of stereoacuity and the conditions that McKee, 1980a), and not be part of a planar structure such as a optimize performance. Nevertheless some kind of qualitative ste- row of dots (Fahle & Westheimer, 1988) or a sheet of small ele- reoscopic sense of depth can be conveyed in many other visual sit- ments (Mitchison & Westheimer, 1984). This phenomenon is uations. As is the case with all spatial tasks, stereoacuity now part of the widely-studied topic called crowding. diminishes with retinal eccentricity. More surprising is the fact 7. Target familiarity and perceptual learning. More than other visual that binocular single vision is not a pre-requisite to stereoscopic thresholds, stereoacuity improves with target familiarity. depth which can also be experienced with certain kinds of diplopic Unlike foveal visual acuity, perceptual learning is the rule rather targets. Conversely, as mentioned above, a reported superimposi- than the exception in measurements for the determination of tion of the right and left-eyed images, even when giving the stereo thresholds (Fendick & Westheimer, 1983) so that the first appearance of fusion, can in some conditions occur without engag- numerical value is not a reliable guide of a subject’s ultimate ing the faculty of stereoscopy. The clinical disambiguation of these ability. Moreover, training on one set of targets does not neces- situations is beyond the scope of this presentation. sarily transfer to others (Coutant, 1993). However, as stereo There can be no stereopsis testing, clinical or otherwise, with- thresholds improve with perceptual learning, some of the more out the instrumentation that permits presentation of either real- physiologically-based performance features, such as crowding space targets minutely differentiated in the third dimension or of or threshold rise with retinal eccentricity, remain proportion- defined images separately to the two eyes, but the primary focus ally invariant (Westheimer & Truong, 1988). for the actual tests has to be on the visual patterns and testing con- ditions. The variable is, of course, disparity and the task is to relate The question of contours is often raised. To begin with, stereop- it to the patient’s response. Researchers often concentrate on the sis, a spatial localizing task even if complicated by the need for proper psychophysical technique for reaching reliable quantitative interocular detection, depends on some sort of differential stimula- data, but in a clinical setting this is more or less taken for granted. tion of neighboring retinal locations, whether or not the words Correction of refractive errors, a staple of optometric practice, uses for an end point a visual acuity determination about which a sea- soned clinician asks no lessons in psychophysical methodology. The same applies to stereopsis, where the decision between the advantages of frequency-of-seeing or staircase methods are best left to the laboratory scientist. 4. Optimal conditions for best stereo performance Just as visual acuity is a determination of the limit of (two- dimensional) spatial discrimination, stereoacuity is a measure- Fig. 2. Three-line stereogram in which the relative placement of the lines in the ment of depth threshold and, as all such tests, should be carried right and left panels cannot be used by even a knowledgeable observer for a reliable out under the conditions that bring forth the best performance. judgment of their depths. Please cite this article in press as: Westheimer, G. Clinical evaluation of stereopsis. Vision Research (2012), http://dx.doi.org/10.1016/j.visres.2012.10.005 4 G. Westheimer / Vision Research xxx (2012) xxx–xxx ‘‘border’’ or contour’’ are applied. The question is how, not disparity associations in favor of the coherent global one, will the whether, the contour is generated. Lines or edges are typical modes feature be evident. Solving a random-dot stereogram requires ste- of marking location, but this can be achieved also by a sequence of reopsis, but in addition a set of higher processes, which initially shorter segments, and indeed it has been demonstrated that some takes time (Harwerth & Rawlings, 1977) but eventually can lead cortical orientation-selective elements in the primate visual cortex to a quick and often instantaneous percept. This means that the respond even to ‘‘illusory’ contours. Hence the emphasis, while pri- negative outcome of a single quick random-dot stereogram test marily on the location of the discontinuity will also be on the im- cannot be accepted as evidence of lack of stereopsis, but a positive age characteristics such as gradient sharpness and magnitude of one is conclusive. Since the individual elements making up the contrast step. panel should be articulated separately, their size, which in most Though a sophisticated and knowledgeable observer might instances determines the minimum disparity, precludes demon- sometimes be able to infer their expected depth from the relative strating superior stereo acuity. placement of the symbols in the right and left panels, there are dis- Pending the availability of 3D computer mechanisms adapted plays in which even this facility will fail (Fig. 2). This is because the for clinical use, the recommended test procedures that minimize spatial operating ranges for two-dimensional locational hyperacu- interacting factors, once the knitting needle check has been satis- ity and for stereoscopic processing differ quite substantially fied, are either (Westheimer & McKee, 1979). (1) A set of translucent plates of a range of thicknesses, with highly-visible emblems on one face except the ones with 5. Clinical testing disparity, which are on the other face (e.g. the Frisby test or a home-made variant), or The consideration in the previous sections imply that, to be (2) A polaroid stereogram with rows of symbols or letters, most effective in the measurement of stereoacuity, targets should slightly jittered in position to preclude guessing, one in each be: row with disparity diminishing from, say, 200–10 arcsec. For near viewing, the interocular position differences would be few in number, in the range of 1–0.05 mm. well articulated, sharply delineated and in good focus binocularly, Under ideal conditions, a trained observer can achieve stereo with high contrast and, at a minimum, medium photopic thresholds as low as 2 arcsec, better even than the best monocular luminance, location hyperacuity. This is not, however, the goal of a clinical test, exposed for at least a good fraction of a second with binocular where 10 arcsec would be a very respectable performance and asynchrony of no more than a few tens of milliseconds. where a normal observer should manifest a reading of better than 1 arcmin on a first test. For reliable clinical measurements they should be minimally encumbered by the influence of prediction, memory and as far as possible, devoid of non-stereoscopic clues to depth. References These conditions are well met by panels, originally part of the The list below is not intended to fully cover the literature on Keystone stereo sets and also included, at least in rudimentary clinical stereo testing, but merely to serve as documentation for form, in the TNO series, modeled after the Snellen visual acuity specific points made in the text. Fortunately, workers in the topic charts. They have several rows of well-separated symbols or let- have available as an invaluable resource the monumental two- ters, of high contrast and minimally 20/40 or 20/60 in size, in volume compendium by Howard (2002). Two more extensive which one symbol or letter in each row of several well-separated reviews (Westheimer, 1994, 2011) deal with related areas of elements has disparity, appearing either in front or back of the oth- stereoscopic vision. ers in its row, with progressively decreasing disparity in sequential rows. To ensure that the very small monocular position differences Coutant, B. E. (1993). Training improvements in human stereoscopic vision. which code for disparity cannot be used for cues, the elements Unpublished Ph.D. thesis. Berkeley: University of California. might be spaced somewhat irregularly in each row (Fig. 3). Fahle, M., & Westheimer, G. (1988). Local and global factors in disparity detection of rows of points. Vision Research, 28(1), 171–178. A class of target that fails to match at least some of these criteria Fendick, M., & Westheimer, G. (1983). Effects of practice and the separation of test is the random dot stereogram. Made of many small tokens, arrayed targets on foveal and peripheral stereoacuity. Vision Research, 23(2), 145–150. in a way so that a subset forming a geometrical feature such as a Harwerth, R. S., & Rawlings, S. C. (1977). Viewing time and stereoscopic threshold circle or square has a disparity, it has the remarkable property of with random-dot stereograms. American Journal of Optometry and Physiological Optics, 54, 452–457. hiding the outline of the feature in the monocular views (Julesz, Howard, H. J. (1919). A test for the judgment of distance. American Journal of 1960). Only to someone who has stereo vision and whose visual Ophthalmology, 2, 656–675. apparatus can disentangle the many small and irrelevant right/left Howard, I. P. (2002). Seeing in depth (2 Vols.). Toronto: I. Porteous. Julesz, B. (1960). Binocular depth perception of computer-generated patterns. Bell Systems Technical Journal, 39, 1125–1162. Mitchison, G. J., & Westheimer, G. (1984). The perception of depth in simple figures. Vision Research, 24(9), 1063–1073. Mueller, C. G., & Lloyd, V. V. (1948). Stereoscopic acuity for various levels of illumination. Proceedings of the National Academy of Sciences of the United States of America, 34, 223–227. Ogle, K. N., & Weil, M. P. (1958). Stereoscopic vision and the duration of the stimulus. A.M.A Archives of Ophthalmology, 59, 4–17. Westheimer, G. (1979). Cooperative neural processes involved in stereoscopic Fig. 3. Sample line of a right/left pair of stereogram panels for a stereoacuity test in acuity. Experimental Brain Research, 36(3), 585–597. which stimulus conditions are optimized. Sharp, clearly separated features are Westheimer, G. (1994). The Ferrier Lecture, 1992. Seeing depth with two eyes: members of ensembles already known to the patient (alphanumeric characters or stereopsis. Proceedings of the Royal Society of London. Series B: Biological Sciences, geometrical shapes). One symbol in each row has disparity and should be 257(1349), 205–214. recognized as either in front or behind the rest. Locations in row are not quite Westheimer, G. (2011). Three-dimensional displays and stereo vision. Proceedings of regular so that lateral position cannot be used as a depth clue. Vertical arraying of the Royal Society of London. Series B: Biological Sciences, 278(1716), 2241–2248. such rows with progressively diminishing disparity would generate a chart that can Westheimer, G., & McKee, S. P. (1979). What prior uniocular processing is necessary be utilized in the manner of the Snellen chart for visual acuity. for stereopsis? Investigative Ophthalmology and Visual Science, 18(6), 614–621. Please cite this article in press as: Westheimer, G. Clinical evaluation of stereopsis. Vision Research (2012), http://dx.doi.org/10.1016/j.visres.2012.10.005 G. Westheimer / Vision Research xxx (2012) xxx–xxx 5 Westheimer, G., & McKee, S. P. (1980a). Stereogram design for testing local Westheimer, G., & Truong, T. T. (1988). Target crowding in foveal and peripheral stereopsis. Investigative Ophthalmology and Visual Science, 19(7), 802–809. stereoacuity. American Journal of Optometry and Physiological Optics, 65(5), Westheimer, G., & McKee, S. P. (1980b). Stereoscopic acuity with defocused and 395–399. spatially filtered retinal iamges. Journal of the Optical Society of America, 70, Wilcox, W. W. (1932). The basis for the dependence of visual acuity on illumination. 772–778. Proceedings of the National Academy of Sciences of the United States of America, 18, Westheimer, G., & Pettet, M. W. (1990). Contrast and duration of exposure 47–56. differentially affect vernier and stereoscopic acuity. Proceedings of the Royal Wist, E., & Gogel, W. C. (1966). The effect of inter-ocular delay and repetition Society of London. Series B: Biological Sciences, 241(1300), 42–46. interval on depth perception. Vision Research, 6, 325–334. Please cite this article in press as: Westheimer, G. Clinical evaluation of stereopsis. Vision Research (2012), http://dx.doi.org/10.1016/j.visres.2012.10.005 Downloaded from bjo.bmj.com on 29 January 2007 Br J Ophthalmol 2003;87:1205 1205 Cover illustration .......................................................................... In the search for stereopsis O wls were probably not the first allow satisfactory diurnal acuity for nocturnal avian denizens. After escape or short flights if disturbed. The all, they had to evolve from a cone acuity is at least 11 cycles per diurnal species, and this probably degree (Snellen equivalent of at least required some bridge families. The 20/50) and probably better (rods are bridge between the sister taxa of swifts more densely packed and probably have and hummingbirds to the nocturnal better nocturnal acuity) with good owls was probably the order motion sensitivity. The frogmouth Caprimulgiformes with good cladistic globes are large and together outweigh and systematic evidence of this inter- the brain. The eyes are tubular in shape mediacy. The strange birds in this order with bony scleral ossicles supporting the are not household names and include anterior segment, much like those of the nightjars, oil birds, potoos, owlet- owls. These large eyes, steep corneas, nightjars, and frogmouths. All of these and large rounded lenses are excellent they are almost impossible to see. When birds are characterised by enormous at light capture to improve their noctur- approached, they almost never move, gaping mouths, tiny feet, crepuscular nal image. Swifts have their eyes lateral but can make a buzzing sound that and/or nocturnal activity. The nightjars, resembles bees, presumably to ward off with large monocular visual fields, but oil birds (with perhaps one rare excep- predators. Their prey as well as their little or no binocular field. Owls have tion), potoos, and swifts are wing predators are fooled by the plumage. the most frontally placed eyes and feeders and do not take food off the When a prey moves into range, the bird probably the largest binocular field of substrate. Only the frogmouths and the attacks and dispatches the prey quickly any avian family. Frogmouths have owlet-nightjars can utilise substrate even using the stiff whisker-like feath- intermediately placed eyes and visual food sources, and appear to be close to ers (see BJO April 2001 for discussion of fields to match. These birds spend most the owls themselves in several ways. The avian feathers) around their beaks to of their time with their visual axes Papuan frogmouth on this month’s cover is a good example of this pre- assist in capture. Their feet are weak confined to a small region of the visual dominately insectivorous order, and will and do not assist in capture, but with space, with little or no movement. The illustrate the evolution from swifts to the large, sturdy, triangular hooked bill, visual axes of the frogmouth are more owls. the feet are unnecessary for the kill. divergent than those of the owl, but The frogmouth family (Podargidae) The yawning oropharynx that gives much less divergent than the swift’s. comprises 12 species and is probably a the bird its odd name apparently has a Owls have little to no ocular motility, relic family close to owls. As can be most curious function. The open mouth but frogmouths can align their visual seen, the Papuan frogmouth resembles exudes an odour that attracts insects. axes to gain stereopsis in a frontal gaze an owl, but has different feeding habits, Nocturnal flying insects will investigate because of better ocular motility, and and hence different visual mechanisms. the source of such a promising scent. they do have a binocular ‘‘Wulst’’ This frogmouth has a large hooked beak This living flytrap, as Jared Diamond (raighly equivalent to the visual cortex used to seize invertebrates, a primary (Natural History 1994;103:4) called it, in mammals) to provide them with true food item. Up to 50 cm in height and may have the ultimate vertebrate seden- stereopsis. Curiously, when frogmouths weighing as much as 550 g, frogmouths tary predation method requiring almost make saccades they are almost always in have a booming call that can be most no hunting skills. It is doubtful, how- opposite directions. This evolutionary disquieting in the Australasian back ever, that enough food can be secured intermediate position between the country that is their home. by this method, so the sallying described swifts and the owls, visually, illustrates Best known for their camouflage above is probably preferred. Many of the the binocular imperative for sophisti- techniques, the frogmouths do not for- other caprimulgids, such as the night cated nocturnal predation. age, or trawl, but rather they sally. hawks, have large mouths so that they So, in the evolution of truly nocturnal Sitting rigidly and still, and resembling may trawl with their mouths open birds, swifts, already crepuscular, prob- a dead branch or stump, the bird will through swarms of insects to secure ably began the process. As the capri- swoop or drop directly to the forest floor food with minimal visual requirements. mulgid order evolved, frogmouths for nocturnal insects, spiders, lizards, or The visual mechanisms of the frog- discovered the substrate larder of insects even mice. Surprisingly, although rela- mouth are interesting and probably and mammals. They became part of the tively swift on the wing, they are poor reflect its intermediate position between bridge that facilitated the emergence of fliers but, then, extended or aerobatic the diurnal swifts and the nocturnal owls into the dark of night. flight isn’t really necessary for their owls. The frogmouths have a predomi- lifestyle. nately rod retina, in contrast to the I R Schwab, N S Hart As mentioned they are known for swifts. None the less, there are at least University of California, Davis, Department of their camouflage, and to find them in three different cone types with three Ophthalmology, Sacramento, CA, USA and Vision Touch and Hearing Research Centre, daylight requires as much luck as skill. different coloured oil droplets (to be University of Queensland, Queensland, These birds are so well camouflaged discussed in a subsequent essay) which Australia;

www.bjophthalmol.com Downloaded from bjo.bmj.com on 29 January 2007 1442 Br J Ophthalmol 2003;87:1442 Cover illustration .............................................................................. Double crossed M ost owls have excellent noctur- tuned neurons (tuned to frequency and nal vision but are helpless in spatial location). These neurons project total, complete darkness. Not so to the optic tectum (analogous to the the remarkable barn owl (Tyto alba). superior colliculus in mammals) result- Differing from the more traditional ing in a spatial map receiving input from owls, barn owls are members of the the eyes and ears. Hence, the auditory family Tytonidae, a small family of birds space aligns with, and is taught by, the with distinctive ear bones. These birds visual system. This animal can ‘‘see’’ with are keen hunters capable of acoustic its ears because these summed inputs location in total darkness without the then project into the ‘‘Wulst’’ (visual help of any photons! forebrain considered analogous to the A nocturnal owl, Tyto alba is one visual cortex of mammals), although of the few birds that are virtually perceptually this would be impossible pandemic in its distribution, inhabiting for us to understand completely. all continents except Antarctica. With The visual system of barn owls has at a distinctive appearance known to least one more clever adaptation. Our most of us, this owl has a heart-shaped brain may recognise objects by form facial disc that funnels sounds to alone, requiring contour recognition. In its paired, but asymmetrically placed, a natural environment, however, con- ear holes. These ear holes allow for tours may be incomplete because of very accurate localisation of sound so other objects, shadows, or camouflage. that the barn owl can actually strike Investigators have shown that barn owls its prey in total darkness. Boasting can indeed make assumptions about several novel anatomical and neurologi- incomplete contours, entitled subjective cal design features, the asymmetric contours, to fill in the gaps. Further- ears and auditory processing of this more, direct neuronal recordings have owl are capable of detecting very documented this ‘‘coding’’ of subjective slight interaural time differences. That Barn owl eye. Note the asymmetrical globe with contours and the process of high order ability enables a most enviable auditory a large equatorial diameter. perceptual interpretation (Nieder A et al, sensory mechanism that provides Nature Neuroscience 1999;2:660–3). Simply accurate azimuth and elevation localisa- put, the target rodents cannot hide in tion, as well as directional motion rapid and dramatic head movements of as much as 270˚ and the ability to turn scattered weeds and foliage as this will detection to an accuracy of approxi- not successfully break up the ‘‘search mately 2–3˚! its head almost upside down. image’’ of the owls. This added layer of Nevertheless, even with its extra- Most birds have a completely crossed chiasm and each eye should be com- sophisticated visual processing permits ordinary hearing, this owl has the typical this bird to be a highly effective excellence in visual acuity found in most pletely, and only, represented in the contralateral visual cortex, but such is nocturnal hunter. owls. The bird has predominantly rod Barn owls occupy various habitats, retina restricting it to a nocturnal life- not the case with barn owls. Pettigrew documented that they have excellent although they are usually a grassland/ style. It has a rather limited range of stereopsis (Pettigrew JD et al, Science marshland species. Most owls are help- accommodation probably with a max- 1976;193:675), which could be guessed ful to human agriculture, but this one is imum of 10 dioptres, although accom- from the frontally placed eyes and exceptionally so, consuming approxi- modation may be used as a distance predatorial lifestyle. He later documen- mately 1K times its weight in rodents clue for this bird. It has large eyes ted that there is a second supraoptic each night during the nesting season with a very large anterior chamber chiasm that allows for approximately and somewhat less during other parts of helping to create a long anterior focal 50% of the ipsilateral fibres to cross the year. Since the bird is not particu- length. This 400 g animal has a cornea the brain a second time allowing for larly fast it must rely upon stealth to that is 12 mm in diameter with an input of each eye to each side of the catch its prey and, indeed, because of axial length of 18 mm. This allows for brain, providing stereopsis equivalent to special adaptations to its feathers along the image to be spread over a wider humans. This second crossing repre- its exposed flight surfaces, it is silent as area on the retina and hence increased sents neurological convergent evolution it approaches dinner. image size. As can be seen in the figure on this page, the eyes are asymmetrical suggesting that stereopsis is essential These awesome visual and auditory and tubular in shape with cartilagi- for predators and can evolve in different predators represent an evolutionary tri- nous/bony supports just anterior to the ways. umph of vertebrate sensory evolution. equator. The extraocular muscles have But vision has a uniquely important I R Schwab atrophied evolutionarily, and are rudi- role for this creature. The owl may rely University of California, Davis, Sacramento, mentary. This means that the owl upon its excellent auditory system, but CA, USA;

cannot move its eyes, and must rely the auditory spatial map is trained Images by the author upon its thin, flexible spinal column visually. Located in the inferior collicu- Thanks to William DeBello for the specimens and strong neck muscles to allow for lus, the auditory maps have sharply to study and his comments on the essay. www.bjophthalmol.com Downloaded from bjo.bmj.com on 29 January 2007 138 Br J Ophthalmol 2007;91:138 Cover illustration ................................................................................... Keeping a cool head S harks always draw a crowd. We have another, and to orient towards or away have evolved, and in different ways, as a macabre fascination with these from that odour. These olfactory abilities these classes diverged. creatures because of their command- almost certainly lead this cartilaginous The retina of S lewini has not been well ing presence and predatory lifestyle. The fish to its prey as hammerheads can studied, but it contains rods and cones in hammerhead shark evokes further inter- detect one part per 25 million of blood sharks such as the lemon shark, Negaprion est because of the bizarre morphology of in sea water. brevirostris, and the silky shark, its head. What purpose would such a The cephalofoil also houses electrore- Carcharhinus falciformis, which have head serve and how does the creature ceptors, called the ‘‘ampullae of between 5:1 and 12:1 rods to cones. manage its sensory input? Lorenzini’’, unique to elasmobranchs Some investigation into another species The class Chondrichthyes arose in the and the chimaera. This unique organ of hammerhead, the bonnet head, shows Silurian approximately 415 million years senses low-level electric current in water, a dorsotemporal visual streak or band of ago, and includes all fish having a cartila- with sampling carried out via pores increased ganglion cells, but it is not ginous skeleton. Evolutionarily, these fish distributed along the dorsal and ventral understood how this is used. Almost followed a different path by separating surfaces of the cephalofoil. The utility of nothing is known about the visual field from teleosts or bony fish. Teleosts even- the electrosensory abilities is poorly of hammerheads and how they combine tually led to tetrapods and terrestrial understood, although prey location and the two different monocular visual fields. creatures (BJO September 2006), but the migration have been proposed. While An interesting, but as yet unstudied, chondrichthyes evolved in a different scything its way through water, a ham- aspect of the hammerhead eye is the direction that now includes the modern merhead processes odours, weak electric anatomy of the optic nerve. The nerve elasmobranchs—sharks, skates and rays. currents and visual inputs, although it is may course a foot or more from the eye to The last common ancestor of both cartila- not clear how these signals are reconciled the brain, as if a stalk, on the dorsal ginous and bony fish probably arose and integrated. aspect of the ‘‘hammer’’ with no cartila- approximately 430 million years ago. That What role does the morphology of the ginous canal or much protection, as can ancestor had a primitive piscine-like eye eye and vision play in the sensory abilities be seen in the computed tomography and first showed a superior oblique muscle of Sphyrna lewini? scan below. The chiasm is completely attaching anteriorly to the globe, a key step Sharks have a surprisingly thin cornea crossed as in bony fish, suggesting that in evolution. Extant cartilaginous and (approximately 160 mm), with epithelium the last common ancestor in the shallow bony fish have similar eyes that possess making up to one third of that thickness Ordovician seas had a similar anatomy. more similarities than differences from with no functioning endothelium. Sharks The optic nerves lead to the optic tectum, each other, providing a window on the maintain corneal clarity with only a analogous to the visual cortex in mam- sequence of ocular evolution. primitive endothelial layer, or none at mals. The tectum also receives auditory, Sometime during the Oligocene (34–24 all, by keeping their corneae compact mechanoreceptive, electroreceptive, million years ago), the hammerhead with perpendicularly oriented collagen somatosensory and trigeminal nerves. It family (Sphyrnidae, having eight species) fibrils or sutural fibres. is not clear how these inputs are inte- arose within the elasmobranchs as a Much like other elasmobranchs, sharks grated, but the high degree of sensory peculiar evolutionary anomaly. The nar- accommodate using a protractor lentis, input suggests that these creatures are rowed, dorsal–ventral flattened head, which pulls the lens away from the retina very much tuned in to their surroundings, termed a ‘‘cephalofoil’’, probably evolved as compared with teleosts (bony fish) that with magnificent sensory perception and for improved sensory perception, but not have a retractor lentis muscle that moves a very cool head. necessarily vision. The unusual cephalo- the lens towards the retina (BJO April Ivan R Schwab, foil design allows for improved stereo- 2006). This suggests that the last common University of California, Davis, Sacramento, olfaction and electroreception. ancestor of sharks and teleosts, mentioned CA, USA;

Hammerheads have among the largest above, probably had rudimentary, if any, D Michelle McComb, olfactory bulb to brain ratios of any accommodation. Accommodation must Florida Atlantic University, Florida, USA species and must rely heavily on this sense. The scalloped hammerhead has an olfactory rosette/bulb that occupies 7% of its total brain mass as compared with approximately 3% for sharks in other families (Kajiura, SM et al J Morp 2005;264:253–63). Hammerheads have special grooves leading to the wide- spaced nares (essentially the nasal pas- sages at the distal tips of the cephalofoil) that lead to enormous olfactory rosettes; thus, these sharks have true stereo- olfaction. This ability is defined as true olfactory tropotaxis, or the ability to Figure 1 Olfactory organs to absorb scent (olfactory rosettes) in Sphyrna lewini as illustrated by white compare odour from one side with arrows, and optic nerves with blue arrows (computed tomography scan with false colour). www.bjophthalmol.com