Prediction Models and Decision Support: Chances and Challenges

Abstract

A clinical prediction model can assist doctors in arriving at the most likely diagnosis or estimating the prognosis. By utilizing various patient- and disease-related properties, such models can yield objective estimations of the risk of a disease or the probability of a certain disease course for individual patients. Doctors can then use this individual probability in their decision making process, thereby (hopefully) improving patient outcome. However, even when its probability estimates are both accurate and valid in ‘new’ patients, a prediction model does not necessarily enhance a doctor’s decision making. In so-called ‘impact studies’ the prediction model is implemented in daily practice and the effects on clinical outcomes are compared to care-as-usual – i.e., care without using the model. A prediction model for postoperative nausea and vomiting (PONV) was implemented in the anesthesiology department of an academic hospital. The research question was whether the prediction model improved doctors decisions as to which patients should be administered preventive drugs as ‘PONV prophylaxis’ during surgery. We also studied whether the prophylaxis indeed reduced PONV. In a large-scale randomized trial, the prediction model was implemented in the form of decision support, which presented predicted PONV probabilities for individual patients to their doctors in the electronic patient record during surgery. In contrast to care-as-usual, doctors who were presented their patients’ PONV probabilities administered more PONV prophylaxis to their patients, particularly to patients with a high predicted risk. However, this positive effect on doctors’ behavior did not result in a corresponding beneficial effect on patient outcome, as there was no difference in the PONV incidence between both study groups. In a second impact study, the format of the decision support tool was changed. The tool now presented an additional treatment recommendation on PONV prophylaxis to doctors. This ‘directive’ format further increased the administration of PONV prophylaxis by doctors. In contrast to the ‘assistive’ randomized trial –probability estimates were presented without recommendations – the increased administration lowered the incidence of PONV. Through interviews and surveys, we concluded that using probability estimates during a decision can be problematic, because the probability estimates need to be consciously integrated with the clinical experience of the doctor to properly assess the pros and cons of treatment. Thus, in a high workload environment – such as an operating room – conscious use of a probability may require extra attention from doctors. In their daily decision making, doctors depend on the context in which patient and problem are presented. Doctors should thus be able to interpret decision support information within the context of the problem. If such interpretation is in any way impeded, decision support can actually be confusing, complicating a doctor’s decision. Therefore, prediction models and decision support should be as closely tailored to the doctors’ workflow as possible, so as to minimize the need for extra attention. It is likely that the joint forces of doctors and technology will produce better decisions than either of them exclusively: after all, they differ in how they handle a decision’s complexity and are thus complementary.