Evaluation of ML-Based Clinical Decision Support Tool to Replace an Existing Tool in an Academic Health System: Lessons Learned.

Abstract

There is increasing application of machine learning tools to problems in healthcare, with an ultimate goal to improve patient safety and health outcomes. When applied appropriately, machine learning tools can augment clinical care provided to patients. However, even if a model has impressive performance characteristics, prospectively evaluating and effectively implementing models into clinical care remains difficult. The primary objective of this paper is to recount our experiences and challenges in comparing a novel machine learning-based clinical decision support tool to legacy, non-machine learning tools addressing potential safety events in the hospitals and to summarize the obstacles which prevented evaluation of clinical efficacy of tools prior to widespread institutional use. We collected and compared safety events data, specifically patient falls and pressure injuries, between the standard of care approach and machine learning (ML)-based clinical decision support (CDS). Our assessment was limited to performance of the model rather than the workflow due to challenges in directly comparing both approaches. We did note a modest improvement in falls with ML-based CDS; however, it was not possible to determine that overall improvement was due to model characteristics.

Department

Description

Provenance

Citation

Published Version (Please cite this version)

10.3390/jpm10030104

Publication Info

Woo, Myung, Brooke Alhanti, Sam Lusk, Felicia Dunston, Stephen Blackwelder, Kay S Lytle, Benjamin A Goldstein, Armando Bedoya, et al. (2020). Evaluation of ML-Based Clinical Decision Support Tool to Replace an Existing Tool in an Academic Health System: Lessons Learned. Journal of personalized medicine, 10(3). p. E104. 10.3390/jpm10030104 Retrieved from https://hdl.handle.net/10161/25610.

This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.

Scholars@Duke

Goldstein

Benjamin Alan Goldstein

Professor of Biostatistics & Bioinformatics

I study the meaningful use of Electronic Health Records data. My research interests sit at the intersection of biostatistics, biomedical informatics, machine learning and epidemiology. I collaborate with researchers both locally at Duke as well as nationally. I am interested in speaking with any students, methodologistis or collaborators interested in EHR data.

Please find more information at: https://sites.duke.edu/bgoldstein/

Bedoya

Armando Diego Bedoya

Assistant Professor of Medicine

Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.