Multisensory perception of spatial orientation and self-motion

Abstract

The aim of this project was to improve our insight in how the brain combines information from different sensory systems (e.g. vestibular and visual system) into an integrated percept of self-motion and spatial orientation. Based on evidence from other research in different areas, such as hand-eye coordination, we hypothesized that the human brain takes the reliability of the individual sensory cues into account when estimating the body’s motion and orientation in the environment. This mechanism is called “maximum likelihood integration” (MLI), in which the brain attributes the most weight to the sensory cues with the least variance. This integration results in a statistically optimal percept: it is the most precise estimate possible given the sensory cues. The results were not always consistent with the hypothesis of MLI. For example, the first two simulator experiments showed little evidence that MLI applies to the integration of visual and vestibular information of the direction of linear self-motion. Multisensory judgments (with visual and vestibular cues available) were not more precise than unisensory judgments (with only visual or vestibular information). One of the contributing factors was the limited realism of the synthetic visual stimulus, which may have caused a perceptual dissociation between the visual and vestibular motion. On the other hand, we did find supporting evidence that information from the two vestibular subsystems, i.e. the semi-circular canals and the otoliths, is combined in a statistically optimal fashion. Normally, tilting the head stimulates both the semi-circular and otoliths together. Hence it is difficult to stimulate the otoliths in isolation. In a carefully designed experiment in the DESDEMONA simulator we created unisensory stimulation of the otoliths by using centrifugation, and unisensory stimulation of the semi-circular canals by rotating the subjects while they were lying supine. The multisensory condition comprised normal self-tilt from an upright position. The results agreed with the MLI model. Finally, we participated in the first European Partial Gravity Parabolic Flight Campaign organized by the European Space Agency (ESA), to study the perception of self-tilt under partial gravity conditions: weightlessness (0g), Lunar gravity (0.16g), and Martian gravity (0.38g). This provided a unique opportunity to manipulate the magnitude of gravity, which on Earth is always fixed. Although the data did not provide conclusive evidence of MLI in the integration of visual and vestibular cues, we did the remarkable finding that gravity must exceed a certain threshold value before it is being recognized as reference for vertical. Below this threshold, gravity may be felt as a force, but does not serve as a reference for orientation. The recognition threshold is individually determined and seems to increase with subjects’ age. The older subjects (40+ years) did no longer recognize the direction of “up” under Martian gravity, whereas the younger subjects (under 40) lost their feeling for “up” when gravity was reduced to Lunar levels. This corresponds to reports of the Apollo astronauts having great difficulties in judging the slope of the terrain while exploring the surface of the Moon