Sensory Continuity Across Eye-movements: Forward to a model of visual perception

Abstract

Vision is the dominant sense of humans. This becomes apparent when considering just how much of our society operates under the assumption of perfect vision. Using an (unaltered) smart-phone or driving a car is virtually impossible without vision. In isolation, our senses are constrained by some limitation due to the physical properties of the information that they detect. Spatial accuracy is somewhat low for the auditory system, and touch is limited to only our immediate vicinity. At first glance, vision provides unparalleled accuracy and the ability to both sense at close range and long range, making it a top contender for our primary sensory system. However, vision does contain less apparent limitations. The first limitation is perhaps the most obvious, as we have some subjective insight in this limitation. By keeping your eye still and shifting the focus of your attention, you may notice that the visual system only provides high resolution vision for input that falls in the center of the retina (fovea). The resolution at which we process visual information is lower in the periphery, due to lower (overall) densities of photoreceptors in those areas. Another limitation of the visual system, is that humans are only aware of a small portion of visual space at any given time. The only portion of visual information that is processed to reach consciousness is the information that receives visual attention. Visual attention acts as a selection mechanism for visual input. Rather than processing the entire visual field at once, we select information both voluntarily and based on salience, which in turn is the information that reaches our awareness. The most critical restriction placed on vision is the need to make eye-movements (saccades). It is during saccades that we largely stop processing visual information. This halt in processing is intentional, as processing the visual image fully would lead to retinal smear, like the smear you see when you take a picture with a camera while moving it. Likely, the visual system solves the problem of retinal smear by reducing visual processing around the time of a saccade. Thus, humans only process a small portion of visual space, only during a limited time when the eye is still, and only in high resolution when it is at the center of vision. Yet, these limitations seem completely at odds with our subjective experience of our own visual perception. This stark contrast in our subjective experience and behavior is often coined as the problem of visual continuity; we process information in discrete snapshots, yet our subjective experience of the world is continuous. To accomplish visual continuity across an eye-movement, the visual system must somehow combine two disparate images (before and after a saccade) and do it in such a way that we have no notion of the lack of information processing during the saccade. I argue that the visual system may accomplish continuity of visual perception across eye-movements through several linking mechanisms which come together in one common model: a forward model of visual perception.