What you see is what you get: Brain mechanisms underlying action-modulated perception

Abstract

In order to successfully execute our actions in daily life, it is beneficial to focus on specific features of the incoming visual information. 'Action-modulated perception' refers to the process of enhancing action-relevant perceptual features during the preparation of an action. For instance, when preparing to grasp an object, action-relevant features of the object (orientation, size) will be enhanced. Similarly, when preparing an eye movement, the intended target location will be enhanced. If the visual system is made 'ready' to better process relevant parts of the incoming visual information, the execution of the intended action will improve. The research presented in this thesis focuses on the effects of action preparation on perception, and its implementation in the brain. This was studied in both the oculomotor and skeletomotor system. The time course of events during eye movement preparation was studied using electro-encephalography (EEG). Early frontal activity was found, followed later by occipital activity, contralateral to the eye movement direction. The amplitudes of the early frontal (likely the frontal eye fields, FEF) and late occipital activity were significantly correlated, indicating that the early FEF activity may drive spatial attention shifts by means of top-down feedback projections to the early visual areas during eye movement preparation. Using the same saccade task, single pulse transcranial magnetic stimulation (TMS) was applied to the FEF during eye movement preparation. This enhanced spatial attention at the eye movement end point, when applied shortly after the go-signal and contralateral to the eye movement direction. This indicates that it is the early FEF activity that may drive top-down feedback to the visual areas, modulating perception at the location to which an eye movement is being prepared. The second part of the thesis demonstrates the effect of action-modulated perception for grasping actions. In a behavioral study, subjects were better able to detect changes in the orientation of a bar that was to be grasped, rather than pointed to. This paradigm was used to investigate the role of the anterior intraparietal area (AIP) in mediating action-modulated perception. AIP, a visuomotor area involved in grasping planning and execution, was stimulated during action preparation. This disrupted the increase in orientation sensitivity with grasping preparation over pointing preparation. This shows that AIP is an essential component in establishing action-modulated perception for grasping actions and implicates AIP as a source of top-down feedback to the early visual areas. This perceptual effect of grasping preparation was further investigated using functional magnetic resonance imaging (fMRI) and multivoxel pattern analysis techniques (MVPA). Subjects performed grasping and pointing actions in the 7T MRI scanner. Results show that the action in preparation can be reliably predicted from activity in early visual areas, indicating that action preparation has a profound effect on early visual processing. In this thesis, two important areas have been identified that contribute to the enhancement of action-relevant perceptual features during action preparation. Taken together, this suggests that the preparation of an action modulates action relevant features through top-down feedback connections, enabling action-modulated perception.