Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study.
Date
2023-06
Journal Title
Journal ISSN
Volume Title
Repository Usage Stats
views
downloads
Citation Stats
Abstract
Objective
The purpose of this study was to evaluate the performance of different supervised machine learning algorithms to predict achievement of minimum clinically important difference (MCID) in neck pain after surgery in patients with cervical spondylotic myelopathy (CSM).Methods
This was a retrospective analysis of the prospective Quality Outcomes Database CSM cohort. The data set was divided into an 80% training and a 20% test set. Various supervised learning algorithms (including logistic regression, support vector machine, decision tree, random forest, extra trees, gaussian naïve Bayes, k-nearest neighbors, multilayer perceptron, and extreme gradient boosted trees) were evaluated on their performance to predict achievement of MCID in neck pain at 3 and 24 months after surgery, given a set of predicting baseline features. Model performance was assessed with accuracy, F1 score, area under the receiver operating characteristic curve, precision, recall/sensitivity, and specificity.Results
In total, 535 patients (46.9%) achieved MCID for neck pain at 3 months and 569 patients (49.9%) achieved it at 24 months. In each follow-up cohort, 501 patients (93.6%) were satisfied at 3 months after surgery and 569 patients (100%) were satisfied at 24 months after surgery. Of the supervised machine learning algorithms tested, logistic regression demonstrated the best accuracy (3 months: 0.76 ± 0.031, 24 months: 0.773 ± 0.044), followed by F1 score (3 months: 0.759 ± 0.019, 24 months: 0.777 ± 0.039) and area under the receiver operating characteristic curve (3 months: 0.762 ± 0.027, 24 months: 0.773 ± 0.043) at predicting achievement of MCID for neck pain at both follow-up time points, with fair performance. The best precision was also demonstrated by logistic regression at 3 (0.724 ± 0.058) and 24 (0.780 ± 0.097) months. The best recall/sensitivity was demonstrated by multilayer perceptron at 3 months (0.841 ± 0.094) and by extra trees at 24 months (0.817 ± 0.115). Highest specificity was shown by support vector machine at 3 months (0.952 ± 0.013) and by logistic regression at 24 months (0.747 ± 0.18).Conclusions
Appropriate selection of models for studies should be based on the strengths of each model and the aims of the studies. For maximally predicting true achievement of MCID in neck pain, of all the predictions in this balanced data set the appropriate metric for the authors' study was precision. For both short- and long-term follow-ups, logistic regression demonstrated the highest precision of all models tested. Logistic regression performed consistently the best of all models tested and remains a powerful model for clinical classification tasks.Type
Department
Description
Provenance
Citation
Permalink
Published Version (Please cite this version)
Publication Info
Park, Christine, Praveen V Mummaneni, Oren N Gottfried, Christopher I Shaffrey, Anthony J Tang, Erica F Bisson, Anthony L Asher, Domagoj Coric, et al. (2023). Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study. Neurosurgical focus, 54(6). p. E5. 10.3171/2023.3.focus2372 Retrieved from https://hdl.handle.net/10161/27925.
This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.
Collections
Scholars@Duke
Oren N Gottfried
I specialize in the surgical management of all complex cervical, thoracic, lumbar, or sacral spinal diseases by using minimally invasive as well as standard approaches for arthritis or degenerative disease, deformity, tumors, and trauma. I have a special interest in the treatment of thoracolumbar deformities, occipital-cervical problems, and in helping patients with complex spinal issues from previously unsuccessful surgery or recurrent disease.I listen to my patients to understand their symptoms and experiences so I can provide them with the information and education they need to manage their disease. I make sure my patients understand their treatment options, and what will work best for their individual condition. I treat all my patients with care and concern – just as I would treat my family. I am available to address my patients' concerns before and after surgery. I aim to improve surgical outcomes for my patients and care of all spine patients with active research evaluating clinical and radiological results after spine surgery with multiple prospective databases. I am particularly interested in prevention of spinal deformity, infections, complications, and recurrent spinal disease. Also, I study whether patient specific variables including pelvic/sacral anatomy and sagittal spinal balance predict complications from spine surgery.
Christopher Ignatius Shaffrey
I have more than 25 years of experience treating patients of all ages with spinal disorders. I have had an interest in the management of spinal disorders since starting my medical education. I performed residencies in both orthopaedic surgery and neurosurgery to gain a comprehensive understanding of the entire range of spinal disorders. My goal has been to find innovative ways to manage the range of spinal conditions, straightforward to complex. I have a focus on managing patients with complex spinal disorders. My patient evaluation and management philosophy is to provide engaged, compassionate care that focuses on providing the simplest and least aggressive treatment option for a particular condition. In many cases, non-operative treatment options exist to improve a patient’s symptoms. I have been actively engaged in clinical research to find the best ways to manage spinal disorders in order to achieve better results with fewer complications.
Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.