Development and Assessment of an Interpretable Machine Learning Triage Tool for Estimating Mortality After Emergency Admissions.



Triage in the emergency department (ED) is a complex clinical judgment based on the tacit understanding of the patient's likelihood of survival, availability of medical resources, and local practices. Although a scoring tool could be valuable in risk stratification, currently available scores have demonstrated limitations.


To develop an interpretable machine learning tool based on a parsimonious list of variables available at ED triage; provide a simple, early, and accurate estimate of patients' risk of death; and evaluate the tool's predictive accuracy compared with several established clinical scores.

Design, setting, and participants

This single-site, retrospective cohort study assessed all ED patients between January 1, 2009, and December 31, 2016, who were subsequently admitted to a tertiary hospital in Singapore. The Score for Emergency Risk Prediction (SERP) tool was derived using a machine learning framework. To estimate mortality outcomes after emergency admissions, SERP was compared with several triage systems, including Patient Acuity Category Scale, Modified Early Warning Score, National Early Warning Score, Cardiac Arrest Risk Triage, Rapid Acute Physiology Score, and Rapid Emergency Medicine Score. The initial analyses were completed in October 2020, and additional analyses were conducted in May 2021.

Main outcomes and measures

Three SERP scores, namely SERP-2d, SERP-7d, and SERP-30d, were developed using the primary outcomes of interest of 2-, 7-, and 30-day mortality, respectively. Secondary outcomes included 3-day mortality and inpatient mortality. The SERP's predictive power was measured using the area under the curve in the receiver operating characteristic analysis.


The study included 224 666 ED episodes in the model training cohort (mean [SD] patient age, 63.60 [16.90] years; 113 426 [50.5%] female), 56 167 episodes in the validation cohort (mean [SD] patient age, 63.58 [16.87] years; 28 427 [50.6%] female), and 42 676 episodes in the testing cohort (mean [SD] patient age, 64.85 [16.80] years; 21 556 [50.5%] female). The mortality rates in the training cohort were 0.8% at 2 days, 2.2% at 7 days, and 5.9% at 30 days. In the testing cohort, the areas under the curve of SERP-30d were 0.821 (95% CI, 0.796-0.847) for 2-day mortality, 0.826 (95% CI, 0.811-0.841) for 7-day mortality, and 0.823 (95% CI, 0.814-0.832) for 30-day mortality and outperformed several benchmark scores.

Conclusions and relevance

In this retrospective cohort study, SERP had better prediction performance than existing triage scores while maintaining easy implementation and ease of ascertainment in the ED. It has the potential to be widely applied and validated in different circumstances and health care settings.






Published Version (Please cite this version)


Publication Info

Xie, Feng, Marcus Eng Hock Ong, Johannes Nathaniel Min Hui Liew, Kenneth Boon Kiat Tan, Andrew Fu Wah Ho, Gayathri Devi Nadarajan, Lian Leng Low, Yu Heng Kwan, et al. (2021). Development and Assessment of an Interpretable Machine Learning Triage Tool for Estimating Mortality After Emergency Admissions. JAMA network open, 4(8). p. e2118467. 10.1001/jamanetworkopen.2021.18467 Retrieved from

This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.



Benjamin Alan Goldstein

Professor of Biostatistics & Bioinformatics

I study the meaningful use of Electronic Health Records data. My research interests sit at the intersection of biostatistics, biomedical informatics, machine learning and epidemiology. I collaborate with researchers both locally at Duke as well as nationally. I am interested in speaking with any students, methodologistis or collaborators interested in EHR data.

Please find more information at:


David Bruce Matchar

Professor of Medicine

My research relates to clinical practice improvement - from the development of clinical policies to their implementation in real world clinical settings. Most recently my major content focus has been cerebrovascular disease. Other major clinical areas in which I work include the range of disabling neurological conditions, cardiovascular disease, and cancer prevention.
Notable features of my work are: (1) reliance on analytic strategies such as meta-analysis, simulation, decision analysis and cost-effectiveness analysis; (2) a balancing of methodological rigor the needs of medical professionals; and (3) dependence on interdisciplinary groups of experts.
This approach is best illustrated by the Stroke Prevention Patient Outcome Research Team (PORT), for which I served as principal investigator. Funded by the AHCPR, the PORT involved 35 investigators at 13 institutions. The Stroke PORT has been highly productive and has led to a stroke prevention project funded as a public/private partnership by the AHCPR and DuPont Pharma, the Managing Anticoagulation Services Trial (MAST). MAST is a practice improvement trial in 6 managed care organizations, focussing on optimizing anticoagulation for individuals with atrial fibrillation.
I serve as consultant in the general area of analytic strategies for clinical policy development, as well as for specific projects related to stroke (e.g., acute stroke treatment, management of atrial fibrillation, and use of carotid endarterectomy.) I have worked with AHCPR (now AHRQ), ACP, AHA, AAN, Robert Wood Johnson Foundation, NSA, WHO, and several pharmaceutical companies.
Key Words: clinical policy, disease management, stroke, decision analysis, clinical guidelines

Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.