Selective cutoff reporting in studies of the accuracy of the Patient Health Questionnaire-9 and Edinburgh Postnatal Depression Scale: Comparison of results based on published cutoffs versus all cutoffs using individual participant data meta-analysis
DEPRESsion Screening Data (DEPRESSD) Collaboration
(2021) International Journal of Methods in Psychiatric Research, volume 30, issue 3, pp.
(Article)
Abstract
Objectives: Selectively reported results from only well-performing cutoffs in diagnostic accuracy studies may bias estimates in meta-analyses. We investigated cutoff reporting patterns for the Patient Health Questionnaire-9 (PHQ-9; standard cutoff 10) and Edinburgh Postnatal Depression Scale (EPDS; no standard cutoff, commonly used 10–13) and compared accuracy estimates based on published
... read more
cutoffs versus all cutoffs. Methods: We conducted bivariate random effects meta-analyses using individual participant data to compare accuracy from published versus all cutoffs. Results: For the PHQ-9 (30 studies, N = 11,773), published results underestimated sensitivity for cutoffs below 10 (median difference: −0.06) and overestimated for cutoffs above 10 (median difference: 0.07). EPDS (19 studies, N = 3637) sensitivity estimates from published results were similar for cutoffs below 10 (median difference: 0.00) but higher for cutoffs above 13 (median difference: 0.14). Specificity estimates from published and all cutoffs were similar for both tools. The mean cutoff of all reported cutoffs in PHQ-9 studies with optimal cutoff below 10 was 8.8 compared to 11.8 for those with optimal cutoffs above 10. Mean for EPDS studies with optimal cutoffs below 10 was 9.9 compared to 11.8 for those with optimal cutoffs greater than 10. Conclusion: Selective cutoff reporting was more pronounced for the PHQ-9 than EPDS.
show less
Download/Full Text
Keywords: Bias, Depressive Disorder, Major, Humans, Patient Health Questionnaire, Psychiatric Status Rating Scales, Sensitivity and Specificity, Journal Article, Meta-Analysis, Research Support, N.I.H., Extramural, Research Support, Non-U.S. Gov't
ISSN: 1049-8931
Publisher: John Wiley and Sons Ltd
Note: Funding Information: This work was supported by the Canadian Institutes of Health Research (CIHR; KRS‐134297, KRS 140994). Ms. Neupane was supported by G.R. Caverhill Fellowship from the Faculty of Medicine, McGill University. Dr. Levis was supported by a CIHR Frederick Banting and Charles Best Canada Graduate Scholarship doctoral award and a Fonds de recherche du Québec—Santé (FRQS) Postdoctoral Award. Mr. Bhandari was supported by a studentship from the Research Institute of the McGill University Health Centre. Dr. Thombs was supported by a Tier 1 Canada Research Chair. Dr. Benedetti was supported by a FRQS researcher salary awards. Dr. Wu was supported by a FRQS Postdoctoral Training Fellowship. Ms. Rice was supported by a Vanier Canada Graduate Scholarship. Ms. Riehm and Ms. Saadat were supported by CIHR Frederick Banting and Charles Best Canada Graduate Scholarship master's awards. Ms. Azar and Mr. Levis were supported by FRQS Masters Training Awards. The primary study by Alvarado et al. was supported by the Ministry of Health of Chile. Collection of data for the study by Arroll et al. was supported by a project grant from the Health Research Council of New Zealand. The primary study by Khamseh et al. was supported by a grant (M‐288) from Tehran University of Medical Sciences. The primary study by Beck et al. was supported by the Patrick and Catherine Weldon Donaghue Medical Research Foundation and the University of Connecticut Research Foundation. The primary study by Bombardier et al. was supported by the Department of Education, National Institute on Disability and Rehabilitation Research, Spinal Cord Injury Model Systems: University of Washington (H133N060033), Baylor College of Medicine (H133N060003), and University of Michigan (H133N060032). Prof. Robertas Bunevicius, MD, PhD (1958‐2016) was Principal Investigator of the primary study by Bunevicius et al, but passed away and was unable to participate in this project. The primary study by Chaudron et al. was supported by a grant from the National Institute of Mental Health (K23 MH64476). Dr. Cholera was supported by a United States National Institute of Mental Health (NIMH) grant (5F30MH096664), and the United States National Institutes of Health (NIH) Office of the Director, Fogarty International Center, Office of AIDS Research, National Cancer Center, National Heart, Blood, and Lung Institute, and the NIH Office of Research for Women's Health through the Fogarty Global Health Fellows Program Consortium (1R25TW00934001) and the American Recovery and Reinvestment Act. Dr. Conwell received support from NIMH (R24MH071604) and the Centers for Disease Control and Prevention (R49 CE002093). The primary study by Couto et al. was supported by the National Counsel of Technological and Scientific Development (CNPq; g444254/2014‐5) and the Minas Gerais State Research Foundation (FAPEMIG; gAPQ‐01954‐14). Collection of data for the primary study by Delgadillo et al. was supported by grant from St. Anne's Community Services, Leeds, United Kingdom. Collection of data for the primary study by Fann et al. was supported by grant RO1 HD39415 from the US National Center for Medical Rehabilitation Research. The primary study by Tissot et al. was supported by the Swiss National Science Foundation (32003B 125493). The primary study by Garcia‐Esteve et al. was supported by grant 7/98 from the Ministerio de Trabajo y Asuntos Sociales, Women's Institute, Spain. Data for the primary study by Gelaye et al. was supported by grant from the NIH (T37 MD001449). The primary study by Inagaki et al. was supported by the Ministry of Health, Labour and Welfare, Japan. The primary study by Twist et al. was funded by the UK National Institute for Health Research under its Programme Grants for Applied Research Programme (RP‐PG‐0606‐1142). The primary study by Phillips et al. was supported by a scholarship from the National Health and Medical and Research Council (NHMRC). The primary study by Liu et al. (2011) was funded by a grant from the National Health Research Institute, Republic of China (NHRI‐EX97‐9706PI). The primary study by Lotrakul et al. was supported by the Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand [49086]. Dr. Bernd Löwe received research grants from Pfizer, Germany, and from the medical faculty of the University of Heidelberg, Germany (project 121/2000) for the study by Gräfe et al. The primary study by Mohd Sidik et al. was funded under the Research University Grant Scheme from Universiti Putra Malaysia, Malaysia and the Postgraduate Research Student Support Accounts of the University of Auckland, New Zealand. The primary study by Nakić Radoš et al. was supported by the Croatian Ministry of Science, Education, and Sports (134‐0000000‐2421). The primary study by Pawlby et al. was supported by a Medical Research Council UK Project Grant (G89292999N). Collection of primary data for the study by Pence et al. was provided by NIMH (R34MH084673). The primary study by Rochat et al. was supported by grants from the University of Oxford (HQ5035), the Tuixen Foundation (9940), the Wellcome Trust (082384/Z/07/Z and 071571), and the American Psychological Association. Dr. Rochat receives salary support from a Wellcome Trust Intermediate Fellowship (211374/Z/18/Z). The primary study by Rooney et al. was funded by the United Kingdom National Health Service Lothian Neuro‐Oncology Endowment Fund. Dr. Stafford received PhD scholarship funding from the University of Melbourne. The primary study by Su et al. was supported by grants from the Department of Health (DOH94F044 and DOH95F022) and the China Medical University and Hospital (CMU94‐105, DMR‐92‐92 and DMR94‐46). The primary study by Tandon et al. was funded by the Thomas Wilson Sanitarium. Collection of data for the studies by Turner et al. (2012) were funded by a bequest from Jennie Thomas through the Hunter Medical Research Institute. The study by van Steenbergen‐Weijenburg et al. was funded by Innovatiefonds Zorgverzekeraars. The primary study by Vega‐Dienstmaier et al. was supported by Tejada Family Foundation, Inc, and Peruvian‐American Endowment, Inc. Dr Vöhringer was supported by the Fund for Innovation and Competitiveness of the Chilean Ministry of Economy, Development and Tourism, through the Millennium Scientific Initiative (IS130005). The primary study by Thombs et al. was done with data from the Heart and Soul Study. The Heart and Soul Study was funded by the Department of Veterans Epidemiology Merit Review Program, the Department of Veterans Affairs Health Services Research and Development service, the National Heart Lung and Blood Institute (R01 HL079235), the American Federation for Aging Research, the Robert Wood Johnson Foundation, and the Ischemia Research and Education Foundation. Collection of data for the primary study by Gjerdingen et al. was supported by grants from the NIMH (R34 MH072925, K02 MH65919, P30 DK50456). No other authors reported funding for primary studies or for their work on the present study. Funding Information: This work was supported by the Canadian Institutes of Health Research (CIHR; KRS-134297, KRS 140994). Ms. Neupane was supported by G.R. Caverhill Fellowship from the Faculty of Medicine, McGill University. Dr. Levis was supported by a CIHR Frederick Banting and Charles Best Canada Graduate Scholarship doctoral award and a Fonds de recherche du Qu?bec?Sant? (FRQS) Postdoctoral Award. Mr. Bhandari was supported by a studentship from the Research Institute of the McGill University Health Centre. Dr.?Thombs was supported by a Tier 1 Canada Research Chair. Dr.?Benedetti was supported by a FRQS researcher salary awards. Dr.?Wu was supported by a FRQS Postdoctoral Training Fellowship. Ms. Rice was supported by a Vanier Canada Graduate Scholarship. Ms. Riehm and Ms. Saadat were supported by CIHR Frederick Banting and Charles Best Canada Graduate Scholarship master's awards. Ms. Azar and Mr. Levis were supported by FRQS Masters Training Awards. The primary study by Alvarado et?al. was supported by the Ministry of Health of Chile. Collection of data for the study by Arroll et?al. was supported by a project grant from the Health Research Council of New Zealand. The primary study by Khamseh et?al. was supported by a grant (M-288) from Tehran University of Medical Sciences. The primary study by Beck et?al. was supported by the Patrick and Catherine Weldon Donaghue Medical Research Foundation and the University of Connecticut Research Foundation. The primary study by Bombardier et?al. was supported by the Department of Education, National Institute on Disability and Rehabilitation Research, Spinal Cord Injury Model Systems: University of Washington (H133N060033), Baylor College of Medicine (H133N060003), and University of Michigan (H133N060032). Prof. Robertas Bunevicius, MD, PhD (1958-2016) was Principal Investigator of the primary study by Bunevicius et?al, but passed away and was unable to participate in this project. The primary study by Chaudron et?al. was supported by a grant from the National Institute of Mental Health (K23 MH64476). Dr. Cholera was supported by a United States National Institute of Mental Health (NIMH) grant (5F30MH096664), and the United States National Institutes of Health (NIH) Office of the Director, Fogarty International Center, Office of AIDS Research, National Cancer Center, National Heart, Blood, and Lung Institute, and the NIH Office of Research for Women's Health through the Fogarty Global Health Fellows Program Consortium (1R25TW00934001) and the American Recovery and Reinvestment Act. Dr. Conwell received support from NIMH (R24MH071604) and the Centers for Disease Control and Prevention (R49 CE002093). The primary study by Couto et?al. was supported by the National Counsel of Technological and Scientific Development (CNPq; g444254/2014-5) and the Minas Gerais State Research Foundation (FAPEMIG; gAPQ-01954-14). Collection of data for the primary study by Delgadillo et?al. was supported by grant from St. Anne's Community Services, Leeds, United Kingdom. Collection of data for the primary study by Fann et?al. was supported by grant RO1 HD39415 from the US National Center for Medical Rehabilitation Research. The primary study by Tissot et?al. was supported by the Swiss National Science Foundation (32003B 125493). The primary study by Garcia-Esteve et?al. was supported by grant 7/98 from the Ministerio de Trabajo y Asuntos Sociales, Women's Institute, Spain. Data for the primary study by Gelaye et?al. was supported by grant from the NIH (T37 MD001449). The primary study by Inagaki et?al. was supported by the Ministry of Health, Labour and Welfare, Japan. The primary study by Twist et?al. was funded by the UK National Institute for Health Research under its Programme Grants for Applied Research Programme (RP-PG-0606-1142). The primary study by Phillips et?al. was supported by a scholarship from the National Health and Medical and Research Council (NHMRC). The primary study by Liu et?al. (2011) was funded by a grant from the National Health Research Institute, Republic of China (NHRI-EX97-9706PI). The primary study by Lotrakul et?al. was supported by the Faculty of Medicine, Ramathibodi Hospital, Mahidol University, Bangkok, Thailand [49086]. Dr. Bernd L?we received research grants from Pfizer, Germany, and from the medical faculty of the University of Heidelberg, Germany (project 121/2000) for the study by Gr?fe et?al. The primary study by Mohd Sidik et?al. was funded under the Research University Grant Scheme from Universiti Putra Malaysia, Malaysia and the Postgraduate Research Student Support Accounts of the University of Auckland, New Zealand. The primary study by Naki? Rado? et?al. was supported by the Croatian Ministry of Science, Education, and Sports (134-0000000-2421). The primary study by Pawlby et?al. was supported by a Medical Research Council UK Project Grant (G89292999N). Collection of primary data for the study by Pence et?al. was provided by NIMH (R34MH084673). The primary study by Rochat et?al. was supported by grants from the?University of Oxford (HQ5035), the Tuixen Foundation (9940), the Wellcome Trust (082384/Z/07/Z and 071571), and the American Psychological Association. Dr. Rochat receives salary support from a Wellcome Trust Intermediate Fellowship (211374/Z/18/Z). The primary study by Rooney et?al. was funded by the United Kingdom National Health Service Lothian Neuro-Oncology Endowment Fund. Dr. Stafford received PhD scholarship funding from the University of Melbourne. The primary study by Su et?al. was supported by grants from the Department of Health (DOH94F044 and DOH95F022) and the China Medical University and Hospital (CMU94-105, DMR-92-92 and DMR94-46). The primary study by Tandon et?al. was funded by the Thomas Wilson Sanitarium. Collection of data for the studies by Turner et?al. (2012) were funded by a bequest from Jennie Thomas through the Hunter Medical Research Institute. The study by van Steenbergen-Weijenburg et?al. was funded by Innovatiefonds Zorgverzekeraars. The primary study by Vega-Dienstmaier et?al. was supported by Tejada Family Foundation, Inc, and Peruvian-American Endowment, Inc. Dr V?hringer was supported by the Fund for Innovation and Competitiveness of the Chilean Ministry of Economy, Development and Tourism, through the Millennium Scientific Initiative (IS130005). The primary study by Thombs et?al. was done with data from the Heart and Soul Study. The Heart and Soul Study was funded by the Department of Veterans Epidemiology Merit Review Program, the Department of Veterans Affairs Health Services Research and Development service, the National Heart Lung and Blood Institute (R01 HL079235), the American Federation for Aging Research, the Robert Wood Johnson Foundation, and the Ischemia Research and Education Foundation. Collection of data for the primary study by Gjerdingen et?al. was supported by grants from the NIMH (R34 MH072925, K02 MH65919, P30 DK50456). No other authors reported funding for primary studies or for their work on the present study. Publisher Copyright: © 2021 The Authors. International Journal of Methods in Psychiatric Research published by John Wiley & Sons Ltd.
(Peer reviewed)
