Using Machine Learning and Structural Neuroimaging to Detect First Episode Psychosis: Reconsidering the Evidence
Vieira, Sandra; Gong, Qi Yong; Pinaya, Walter H.L.; Scarpazza, Cristina; Tognin, Stefania; Crespo-Facorro, Benedicto; Tordesillas-Gutierrez, Diana; Ortiz-García, Victor; Setien-Suero, Esther; Scheepers, Floortje E.; Van Haren, Neeltje E.M.; Marques, Tiago R.; Murray, Robin M.; David, Anthony; Dazzan, Paola; McGuire, Philip; Mechelli, Andrea
(2020) Schizophrenia Bulletin, volume 46, issue 1, pp. 17 - 26
(Article)
Abstract
Despite the high level of interest in the use of machine learning (ML) and neuroimaging to detect psychosis at the individual level, the reliability of the findings is unclear due to potential methodological issues that may have inflated the existing literature. This study aimed to elucidate the extent to which
... read more
the application of ML to neuroanatomical data allows detection of first episode psychosis (FEP), while putting in place methodological precautions to avoid overoptimistic results. We tested both traditional ML and an emerging approach known as deep learning (DL) using 3 feature sets of interest: (1) surface-based regional volumes and cortical thickness, (2) voxel-based gray matter volume (GMV) and (3) voxel-based cortical thickness (VBCT). To assess the reliability of the findings, we repeated all analyses in 5 independent datasets, totaling 956 participants (514 FEP and 444 within-site matched controls). The performance was assessed via nested cross-validation (CV) and cross-site CV. Accuracies ranged from 50% to 70% for surfaced-based features; from 50% to 63% for GMV; and from 51% to 68% for VBCT. The best accuracies (70%) were achieved when DL was applied to surface-based features; however, these models generalized poorly to other sites. Findings from this study suggest that, when methodological precautions are adopted to avoid overoptimistic results, detection of individuals in the early stages of psychosis is more challenging than originally thought. In light of this, we argue that the current evidence for the diagnostic value of ML and structural neuroimaging should be reconsidered toward a more cautious interpretation.
show less
Download/Full Text
Keywords: multivariate pattern recognition/classification, neuroimaging/multi-site, psychosis, Psychiatry and Mental health, Research Support, Non-U.S. Gov't, Journal Article
ISSN: 0586-7614
Publisher: Oxford University Press
Note: Funding Information: This work was supported by the European Commission (PSYSCAN—Translating neuroimaging findings from research into clinical practice; 603196 to P.M.); International Cooperation and Exchange of the National Natural Science Foundation of China (81220108013 to Q.G. and A.M.); Wellcome Trust’s Innovator Award (208519/Z/17/Z to A.M.); Foundation for Science and Technology (SFRH/BD/103907/2014 to S.V.), and São Paulo Research Foundation (FAPESP) (Brazil; 2013/05168-7 to W.H.L.P.). The authors have declared that there are no conflicts of interest in relation to the subject of this study. Funding Information: This work was supported by the European Commission (PSYSCAN-Translating neuroimaging findings from research into clinical practice; 603196 to P.M.); International Cooperation and Exchange of the National Natural Science Foundation of China (81220108013 to Q.G. and A.M.); Wellcome Trust's Innovator Award (208519/Z/17/Z to A.M.); Foundation for Science and Technology (SFRH/BD/103907/2014 to S.V.), and S?o Paulo Research Foundation (FAPESP) (Brazil; 2013/05168-7 to W.H.L.P.). The authors have declared that there are no conflicts of interest in relation to the subject of this study. Publisher Copyright: © The Author(s) 2019. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted reuse, distribution, and reproduction in any medium, provided the original work is properly cited. Copyright: Copyright 2021 Elsevier B.V., All rights reserved.
(Peer reviewed)