Methodological quality assessment tools for diagnosis and prognosis research: overview and guidance

Abstract

Background and Objectives: Multiple tools exist for assessing the methodological quality of diagnosis and prognosis research. It can be challenging to decide on when to use which tool. We aimed to provide an overview of existing methodological quality assessment (QA) tools for diagnosis and prognosis studies, highlight the overlap and differences among these tools, and to provide guidance for choosing the appropriate tool. Study Design and Setting: We performed a methodological review of tools designed for assessing risk of bias, applicability, or other aspects related to methodological quality in studies investigating tests/factors/markers/models for classifying or predicting a current (diagnosis) and/or future (prognosis) health state. Tools focusing exclusively on causal research or on reporting quality were excluded. Guidance was subsequently developed to assist in choosing an appropriate QA tool. Results: We identified 14 QA tools, eight of which were developed for assessment of diagnosis studies, four for prognosis studies, and two addressing both. We propose a set of five questions to help guide the process of choosing a QA tool based on the purpose or question of the user: whether the focus is on (1) diagnosis, prognosis, or another domain; (2) a prediction model vs a test/factor/marker; (3) evaluating simply the performance of a test/factor/marker vs assessing its added value over other variables; (4) comparing two or more tests/factors/markers/models; and (5) whether the user aims to assess only risk of bias or also other quality aspects. Conclusion: Existing QA tools for appraising diagnosis and prognosis studies vary in purpose, scope, and contents. Our guidance may help researchers, systematic reviewers, health policy makers, and guideline developers in specifying their purpose and question to select the most appropriate QA tool for their assessment. Plain Language Summary: Methodological quality assessment (QA) tools provide a set of criteria to evaluate how well a medical study was done and how trustworthy its results are. To accurately assess a study's quality, it is important to use a QA tool that matches the type of medical study. However, with many QA tools available for different study types, choosing the right one can be challenging, especially for diagnosis and prognosis studies (ie, studies that evaluate tests, factors, markers, and models used for diagnosis and prognosis). To assist in selecting the best QA tools for diagnostic and prognostic studies, we created an overview of available tools and practical tips for choosing the most appropriate one. After searching online databases and consulting experts in the field, we identified 14 QA tools specific to diagnostic and prognostic studies. Additionally, we developed five key questions to guide users in choosing the best tool for their study. While the 14 QA tools differ in their focus and content, our guidance simplifies the process of choosing the right tool and helps users refine their research question.