The influence of joint attention and partner trustworthiness on cross-modal sensory cueing

Abstract

Joint attention refers to the coordinated attention between social partners to an object of shared interest, usually involving shared gaze toward the object. In the laboratory, however, joint attention is often investigated using computerized gaze cueing tasks that do not allow shared gaze. Instead, these computerized tasks require the participant to maintain fixation on the virtual partner's face, while the partner gazes to the left or right. Here we designed a modified gaze cueing task that better simulates a natural joint attention episode by allowing shared gaze, while still maintaining tight experimental control. In our computerized task the participant's gaze and the gaze of a virtual partner were manipulated independently, resulting in shared or unshared gaze. Following each gaze shift of the virtual partner a touch stimulus was delivered on one of the cheeks of the participant. We analyzed behavioral and neural (electro-encephalography) responses to the touch. Faster reaction-times and stronger lateralization of alpha power were observed when the touched cheek was in a jointly attended hemispace compared with a singly attended or unattended hemispace. Importantly, these effects were unique to joint attention and could not be explained as the additive effects of own gaze and gaze cue direction. Underlining its social nature, we found that the behavioral effect was absent when we repeated our experiment with nonsocial cues (arrows) instead of gaze cues. Furthermore, when we compared trustworthy with untrustworthy virtual partners (trustworthiness judgements based on facial appearance) we found the effect only for trustworthy and not for untrustworthy virtual partners. We conclude that joint attention based on shared gaze influences attentional orienting such that cross-modal sensory processing at the jointly attended location is facilitated, particularly when the partner is trustworthy. This indicates that social interactions and trustworthiness judgements affect cortical and behavioral responses to sensory information.