Vision in context: from behavior to neurons

Abstract

The visual system receives an enormous amount of information from the retina. For us to be able to make sense of our environment, the visual system needs to interpret the incoming information. This interpretation is not only based on the sensory information of the stimulus, but is also based on our knowledge of the world. Using this prior knowledge, the visual system derives the most likely cause of the retinal image. How the retinal image is processed in our brain and how this influences our behavior is a big question in neuroscience. The aim of this dissertation was to investigate how visual perception is affected by both the sensory-driven and knowledge-based information. We describe a number of studies where we investigated how these information sources affect the neural signal (Chapter 3, 4 and 5) and our behavior (Chapter 2). For this, we used a combination of behavioral methods and computational neuroimaging techniques. In our experiments, we used both synthetic and natural images. Where the synthetic images contained mainly sensory-driven information, the natural images contained both sensory-driven and knowledge-based information. We quantified both local contrast energy (sensory-driven information) and subjective importance (knowledge-based information) of the natural images. Using fMRI we can estimate the region of visual space to which each cortical location responds, i.e. the population receptive field (pRF) (Dumoulin & Wandell, 2008). In this thesis, we present three new analysis techniques that extend the analysis of the pRF-method, to measure contextual influences of both sensory-driven and knowledge-based information in visual cortex (Chapter 3, 4 and 5). In chapter 3 we present a method to reconstruct center-surround configurations in human visual cortex. We extended the pRF model by adding a suppressive surround to the pRF. In chapter 4 we present a method to identify (natural) images based on the inherent contrast of the images. In this study, we see that similar images were harder to identify than other images using the fMRI responses of visual area V1. We suggest that this is caused by the internal representation of V1 not only capturing sensory-driven information, but also knowledge-based information. This is supported by our results in chapter 5, where we show that the sensory-driven information (defined by the response to contrast) of early visual areas is enhanced according to the knowledge-based information (defined by subjective importance). In chapter 2 we show that the interaction we found in the neural responses of early visual areas is also reflected in our perceptual functioning, and more specifically in our ability to detect changes in a visual scene. The experiments in this thesis provide a quantitative link between physical sensory stimulation and our subjective perceptual experience on the neural signal and our behavior. Where these effects are usually studied using synthetic images, we extend these results to natural images.