Visualizing Mental Representations of Emotional Faces in Schizophrenia

Abstract

Background: The ability to perceive, recognize and process own and others’ emotions is crucial for efficient and effective social communication. Many different tasks have been used to investigate impairments herein in patients with schizophrenia. Evidence suggests that perception, discrimination and recognition of affective facial expressions are impaired in schizophrenia patients (Green, Horan & Lee, 2015; Chan et al 2010). Importantly, not everyone may interpret the same facial expression similarly. People match their internal representation of specific facial expressions to perceived faces and variation in these internal representations may result in distortions of social reality. The impairments in face and/or emotion processing and the bias toward a more negative experience may be causally related to degradation of the internal representation itself or to disturbances in the higher order evaluation of visual input against functionally intact internal representations. Methods: We use a data-driven technique of psychophysical reverse correlation, which makes it possible to visualize internal representations on computer screens. Participants judge noisy images of faces that are created by superimposing random noise on a single constant base face. The random noise distorts the base face at the pixel level, generating facial variation across stimuli that is fully unconstrained and unaffected by researchers’ a priori expectations. The participants’ responses to a large number of faces are then used to model the facial information that was idiosyncratically diagnostic for the judgments. This analysis yields a classification image (CI) for each participant, which visualizes the facial characteristics that drive judgments of emotional expressions (i.e., their internal representation). Specifically, we used reverse correlation image classification (RCIC) to investigate and reconstruct the mental representation of trustworthiness as expressed on the face. Results: Thus far, we have reliable data from 11 patients and 9 controls. While our results are preliminary, they show that patients are capable of performing the task adequately. Both a visual inspection of the group-level visualizations of the 3 constructs probed (i.e., untrustworthy, neutral, and trustworthy) as well as the computed correlations between these visualizations suggest that the internal representation of these constructs is less expressive in patients when compared to controls. Conclusion: Being able to visualize mental representations of trustworthy and untrustworthy faces in patients with schizophrenia opens up the possibility of further use of RCIC tasks in the investigation of emotion-processing deficiencies in schizophrenia.