Critical Review of Current Approaches for Echocardiographic Reproducibility and Reliability Assessment in Clinical Research.
Date
2016-12
Journal Title
Journal ISSN
Volume Title
Repository Usage Stats
views
downloads
Citation Stats
Abstract
Background
There is no broadly accepted standard method for assessing the quality of echocardiographic measurements in clinical research reports, despite the recognized importance of this information in assessing the quality of study results.Methods
Twenty unique clinical studies were identified reporting echocardiographic data quality for determinations of left ventricular (LV) volumes (n = 13), ejection fraction (n = 12), mass (n = 9), outflow tract diameter (n = 3), and mitral Doppler peak early velocity (n = 4). To better understand the range of possible estimates of data quality and to compare their utility, reported reproducibility measures were tabulated, and de novo estimates were then calculated for missing measures, including intraclass correlation coefficient (ICC), 95% limits of agreement, coefficient of variation (CV), coverage probability, and total deviation index, for each variable for each study.Results
The studies varied in approaches to reproducibility testing, sample size, and metrics assessed and values reported. Reported metrics included mean difference and its SD (n = 7 studies), ICC (n = 5), CV (n = 4), and Bland-Altman limits of agreement (n = 4). Once de novo estimates of all missing indices were determined, reasonable reproducibility targets for each were identified as those achieved by the majority of studies. These included, for LV end-diastolic volume, ICC > 0.95, CV < 7%, and coverage probability > 0.93 within 30 mL; for LV ejection fraction, ICC > 0.85, CV < 8%, and coverage probability > 0.85 within 10%; and for LV mass, ICC > 0.85, CV < 10%, and coverage probability > 0.60 within 20 g.Conclusions
Assessment of data quality in echocardiographic clinical research is infrequent, and methods vary substantially. A first step to standardizing echocardiographic quality reporting is to standardize assessments and reporting metrics. Potential benefits include clearer communication of data quality and the identification of achievable targets to benchmark quality improvement initiatives.Type
Department
Description
Provenance
Citation
Permalink
Published Version (Please cite this version)
Publication Info
Crowley, Anna Lisa, Eric Yow, Huiman X Barnhart, Melissa A Daubert, Robert Bigelow, Daniel C Sullivan, Michael Pencina, Pamela S Douglas, et al. (2016). Critical Review of Current Approaches for Echocardiographic Reproducibility and Reliability Assessment in Clinical Research. Journal of the American Society of Echocardiography : official publication of the American Society of Echocardiography, 29(12). pp. 1144–1154.e7. 10.1016/j.echo.2016.08.006 Retrieved from https://hdl.handle.net/10161/22519.
This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.
Collections
Scholars@Duke
Anna Lisa Chamis
Melissa Anne Daubert
Daniel Carl Sullivan
Research interests are in oncologic imaging, especially the clinical evaluation and validation of imaging biomarkers for therapeutic response assessment.
Michael J Pencina
Michael J. Pencina, PhD
Chief Data Scientist, Duke Health
Vice Dean for Data Science
Director, Duke AI Health
Professor, Biostatistics & Bioinformatics
Duke University School of Medicine
Michael J. Pencina, PhD, is Duke Health's chief data scientist and serves as vice dean for data science, director of Duke AI Health, and professor of biostatistics and bioinformatics at the Duke University School of Medicine. His work bridges the fields of data science, health care, and AI, contributing to Duke’s national leadership in trustworthy health AI.
Dr. Pencina partners with key leaders to develop data science strategies for Duke Health that span and connect academic research and clinical care. As vice dean for data science, he develops and implements quantitative science strategies to support the School of Medicine’s missions in education and training, laboratory and clinical science, and data science.
He co-founded and co-leads the national Coalition for Health AI (CHAI), a multi-stakeholder effort whose mission is to increase trustworthiness of AI by developing guidelines to drive high-quality health care through the adoption of credible, fair, and transparent health AI systems. He also spearheaded the establishment and co-chairs Duke Health’s Algorithm-Based Clinical Decision Support (ABCDS) Oversight Committee and serves as co-director of Duke’s Collaborative to Advance Clinical Health Equity (CACHE).
Dr. Pencina is an internationally recognized authority in the evaluation of AI algorithms. Guideline groups rely on his work to advance best practices for the application of clinical decision support tools in health delivery. He interacts frequently with investigators from academic and industry institutions as well as government officials. Since 2014, he has been acknowledged annually by Thomson Reuters/Clarivate Analytics as one of the world’s "highly cited researchers" in clinical medicine and social sciences, with over 400 publications cited over 100,000 times. He serves as a deputy editor for statistics at JAMA-Cardiology.
Dr. Pencina joined the Duke University faculty in 2013, and served as director of biostatistics for the Duke Clinical Research Institute until 2018. Previously, he was an associate professor in the Department of Biostatistics at Boston University and the Framingham Heart Study, and director of statistical consulting at the Harvard Clinical Research Institute. He received his PhD in Mathematics and Statistics from Boston University in 2003 and holds master’s degrees from the University of Warsaw in actuarial mathematics and business culture.
Email: michael.pencina@duke.edu
Web Sites: medschool.duke.edu; aihealth.duke.edu; https://scholars.duke.edu/person/michael.pencina
Phone: 919.613.9066
Address: Duke University School of Medicine; 2424 Erwin Road, Suite 903; Durham, NC 27705
Pamela Susan Douglas
Pamela S Douglas MD is the Ursula Geller Professor of Research in Cardiovascular Diseases in the Department of Medicine at Duke University and Director of the Multimodality Imaging Program at Duke Clinical Research Institute. During her 30+ years of experience she has led several landmark multicenter government studies and pivotal industry clinical trials along with outcomes research studies. She is renowned for her scientific and policy work in improving the quality and appropriateness of imaging in clinical care, clinical trials and registries and through development and dissemination of national standards for imaging utilization, informatics and analysis. She has been among the pioneers in a number of areas including heart disease in women, sports cardiology, and cardio-oncology. Dr. Douglas’ wealth of experience includes authorship of over 400 peer reviewed manuscripts and 30 practice guidelines, and service as the President of the American College of Cardiology, President of the American Society of Echocardiography, and Chief of Cardiology at both the University of Wisconsin and Duke University. She has also previously served on the faculties of the University of Pennsylvania and Harvard University. She currently serves on the External Advisory Council of the National Heart, Lung and Blood Institute and the Scientific Advisory Board of the Patient Advocate Foundation.
Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.