Incorporating informatively collected laboratory data from EHR in clinical prediction models.

Loading...
Thumbnail Image

Date

2024-07

Journal Title

Journal ISSN

Volume Title

Repository Usage Stats

6
views
6
downloads

Citation Stats

Abstract

Background

Electronic Health Records (EHR) are widely used to develop clinical prediction models (CPMs). However, one of the challenges is that there is often a degree of informative missing data. For example, laboratory measures are typically taken when a clinician is concerned that there is a need. When data are the so-called Not Missing at Random (NMAR), analytic strategies based on other missingness mechanisms are inappropriate. In this work, we seek to compare the impact of different strategies for handling missing data on CPMs performance.

Methods

We considered a predictive model for rapid inpatient deterioration as an exemplar implementation. This model incorporated twelve laboratory measures with varying levels of missingness. Five labs had missingness rate levels around 50%, and the other seven had missingness levels around 90%. We included them based on the belief that their missingness status can be highly informational for the prediction. In our study, we explicitly compared the various missing data strategies: mean imputation, normal-value imputation, conditional imputation, categorical encoding, and missingness embeddings. Some of these were also combined with the last observation carried forward (LOCF). We implemented logistic LASSO regression, multilayer perceptron (MLP), and long short-term memory (LSTM) models as the downstream classifiers. We compared the AUROC of testing data and used bootstrapping to construct 95% confidence intervals.

Results

We had 105,198 inpatient encounters, with 4.7% having experienced the deterioration outcome of interest. LSTM models generally outperformed other cross-sectional models, where embedding approaches and categorical encoding yielded the best results. For the cross-sectional models, normal-value imputation with LOCF generated the best results.

Conclusion

Strategies that accounted for the possibility of NMAR missing data yielded better model performance than those did not. The embedding method had an advantage as it did not require prior clinical knowledge. Using LOCF could enhance the performance of cross-sectional models but have countereffects in LSTM models.

Department

Description

Provenance

Citation

Published Version (Please cite this version)

10.1186/s12911-024-02612-1

Publication Info

Sun, Minghui, Matthew M Engelhard, Armando D Bedoya and Benjamin A Goldstein (2024). Incorporating informatively collected laboratory data from EHR in clinical prediction models. BMC medical informatics and decision making, 24(1). p. 206. 10.1186/s12911-024-02612-1 Retrieved from https://hdl.handle.net/10161/31716.

This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.

Scholars@Duke

Engelhard

Matthew M. Engelhard

Assistant Professor of Biostatistics & Bioinformatics

Developing new machine learning methods for multi-modal longitudinal clinical data to support clinical decision-making.

Bedoya

Armando Diego Bedoya

Associate Professor of Medicine

Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.