Quantifying outcome misclassification in multi-database studies: The case study of pertussis in the ADVANCE project
Gini, Rosa; Dodd, Caitlin N; Bollaerts, Kaatje; Bartolini, Claudia; Roberto, Giuseppe; Huerta-Alvarez, Consuelo; Martín-Merino, Elisa; Duarte-Salles, Talita; Picelli, Gino; Tramontan, Lara; Danieli, Giorgia; Correa, Ana; McGee, Chris; Becker, Benedikt F H; Switzer, Charlotte; Gandhi-Banga, Sonja; Bauwens, Jorgen; van der Maas, Nicoline A T; Spiteri, Gianfranco; Sdona, Emmanouela; Weibel, Daniel; Sturkenboom, Miriam
(2020) Vaccine, volume 38 Suppl 2, issue 2, pp. B56 - B64
(Article)
Abstract
BACKGROUND: The Accelerated Development of VAccine beNefit-risk Collaboration in Europe (ADVANCE) is a public-private collaboration aiming to develop and test a system for rapid benefit-risk (B/R) monitoring of vaccines using European healthcare databases. Event misclassification can result in biased estimates. Using different algorithms for identifying cases of Bordetella pertussis (BorPer)
... read more
infection as a test case, we aimed to describe a strategy to quantify event misclassification, when manual chart review is not feasible. METHODS: Four participating databases retrieved data from primary care (PC) setting: BIFAP: (Spain), THIN and RCGP RSC (UK) and PEDIANET (Italy); SIDIAP (Spain) retrieved data from both PC and hospital settings. BorPer algorithms were defined by healthcare setting, data domain (diagnoses, drugs, or laboratory tests) and concept sets (specific or unspecified pertussis). Algorithm- and database-specific BorPer incidence rates (IRs) were estimated in children aged 0-14 years enrolled in 2012 and 2014 and followed up until the end of each calendar year and compared with IRs of confirmed pertussis from the ECDC surveillance system (TESSy). Novel formulas were used to approximate validity indices, based on a small set of assumptions. They were applied to approximately estimate positive predictive value (PPV) and sensitivity in SIDIAP. RESULTS: The number of cases and the estimated BorPer IRs per 100,000 person-years in PC, using data representing 3,173,268 person-years, were 0 (IR = 0.0), 21 (IR = 4.3), 21 (IR = 5.1), 79 (IR = 5.7), and 2 (IR = 2.3) in BIFAP, SIDIAP, THIN, RCGP RSC and PEDIANET respectively. The IRs for combined specific/unspecified pertussis were higher than TESSy, suggesting that some false positives had been included. In SIDIAP the estimated IR was 45.0 when discharge diagnoses were included. The sensitivity and PPV of combined PC specific and unspecific diagnoses for BorPer cases in SIDIAP were approximately 85% and 72%, respectively. CONCLUSION: Retrieving BorPer cases using only specific concepts has low sensitivity in PC databases, while including cases retrieved by unspecified concepts introduces false positives, which were approximately estimated to be 28% in one database. The share of cases that cannot be retrieved from a PC database because they are only seen in hospital was approximately estimated to be 15% in one database. This study demonstrated that quantifying the impact of different event-finding algorithms across databases and benchmarking with disease surveillance data can provide approximate estimates of algorithm validity.
show less
Download/Full Text
Keywords: Event misclassification, Event-finding algorithms, Incidence of pertussis, Positive predictive value, Molecular Medicine, General Immunology and Microbiology, General Veterinary, Public Health, Environmental and Occupational Health, Infectious Diseases
ISSN: 0264-410X
Publisher: Elsevier BV
Note: Funding Information: Caitlin Dodd, Kaatje Bollaerts, Claudia Bartolini, Giuseppe Roberto, Consuelo Huerta-Alvarez, Elisa Martín-Merino, Talita Duarte-Salles, Gino Picelli, Lara Tramontan, Giorgia Danieli, Ana Correa, Chris McGee, Benedikt Becker, Charlotte Switzer, Jorgen Bauwens, Nicoline van der Maas, Gianfranco Spiteri, Emmanouela Sdona declared no conflicts of interest. Rosa Gini declared that her institution participates in studies funded by Novartis, Eli Lilly, Daiichi Sankyo, compliant with the ENCePP Code of Conduct. Sonja Gandhi-Banga declared that she works for Sanofi Pasteur and holds company shares. Daniel Weibel declared that he has received personal fees from GSK for work unrelated to the submitted work. Miriam Sturkenboom declared that she has received grants from Novartis, CDC and Bill & Melinda Gates Foundation for work unrelated to the submitted work. Funding Information: The Innovative Medicines Initiative Joint Undertaking funded this project under ADVANCE grant agreement no. 115557 , resources of which were composed of a financial contribution from the European Union's Seventh Framework Programme ( FP7/2007-2013 ) and in kind contributions from EFPIA member companies. Funding Information: The authors medical writing and editorial assistance from Margaret Haugh (MediCom Consult, Villeurbanne, France). The results described in this publication are from the proof of concept studies conducted as part of the IMI ADVANCE project with the aim of testing the methodological aspects of the design, conduct and reporting of studies for vaccine benefit-risk monitoring activities. The results presented relate solely to the methodological testing and are not intended to inform regulatory or clinical decisions on the benefits and risks of the exposures under investigation. This warning should accompany any use of the results from these studies and they should be used accordingly. The views expressed in this article are the personal views of the authors and should not be understood or quoted as being made on behalf of or reflecting the position of the agencies or organisations with which the authors are affiliated. The Innovative Medicines Initiative Joint Undertaking funded this project under ADVANCE grant agreement no. 115557, resources of which were composed of a financial contribution from the European Union's Seventh Framework Programme (FP7/2007-2013) and in kind contributions from EFPIA member companies. Caitlin Dodd, Kaatje Bollaerts, Claudia Bartolini, Giuseppe Roberto, Consuelo Huerta-Alvarez, Elisa Mart?n-Merino, Talita Duarte-Salles, Gino Picelli, Lara Tramontan, Giorgia Danieli, Ana Correa, Chris McGee, Benedikt Becker, Charlotte Switzer, Jorgen Bauwens, Nicoline van der Maas, Gianfranco Spiteri, Emmanouela Sdona declared no conflicts of interest. Rosa Gini declared that her institution participates in studies funded by Novartis, Eli Lilly, Daiichi Sankyo, compliant with the ENCePP Code of Conduct. Sonja Gandhi-Banga declared that she works for Sanofi Pasteur and holds company shares. Daniel Weibel declared that he has received personal fees from GSK for work unrelated to the submitted work. Miriam Sturkenboom declared that she has received grants from Novartis, CDC and Bill & Melinda Gates Foundation for work unrelated to the submitted work. Publisher Copyright: © 2019 The Authors
(Peer reviewed)