Evaluation of ML-Based Clinical Decision Support Tool to Replace an Existing Tool in an Academic Health System: Lessons Learned.

Woo, Myung

Alhanti, Brooke

Lusk, Sam

Dunston, Felicia

Blackwelder, Stephen

Lytle, Kay S

Goldstein, Benjamin A

Bedoya, Armando

2022-08-26T13:25:09Z

2022-08-26T13:25:09Z

2020-08-27

2022-08-26T13:25:08Z

There is increasing application of machine learning tools to problems in healthcare, with an ultimate goal to improve patient safety and health outcomes. When applied appropriately, machine learning tools can augment clinical care provided to patients. However, even if a model has impressive performance characteristics, prospectively evaluating and effectively implementing models into clinical care remains difficult. The primary objective of this paper is to recount our experiences and challenges in comparing a novel machine learning-based clinical decision support tool to legacy, non-machine learning tools addressing potential safety events in the hospitals and to summarize the obstacles which prevented evaluation of clinical efficacy of tools prior to widespread institutional use. We collected and compared safety events data, specifically patient falls and pressure injuries, between the standard of care approach and machine learning (ML)-based clinical decision support (CDS). Our assessment was limited to performance of the model rather than the workflow due to challenges in directly comparing both approaches. We did note a modest improvement in falls with ML-based CDS; however, it was not possible to determine that overall improvement was due to model characteristics.

jpm10030104

2075-4426

2075-4426

https://hdl.handle.net/10161/25610

eng

MDPI AG

Journal of personalized medicine

10.3390/jpm10030104

artificial intelligence

clinical decision support

falls

intervention

machine learning

pressure injury

prevention

Evaluation of ML-Based Clinical Decision Support Tool to Replace an Existing Tool in an Academic Health System: Lessons Learned.

Journal article

Alhanti, Brooke|0000-0003-4243-8062

Lytle, Kay S|0000-0001-9845-1501

Goldstein, Benjamin A|0000-0001-5261-3632

Bedoya, Armando|0000-0001-6496-7024

E104

3

Duke

School of Medicine

Staff

Basic Science Departments

Clinical Science Departments

Institutes and Centers

Biostatistics & Bioinformatics

Medicine

Pediatrics

Medicine, Pulmonary, Allergy, and Critical Care Medicine

Duke Clinical Research Institute

Population Health Sciences

Pediatrics, Children's Health Discovery Institute

Published

10