Unachievable Region in Precision-Recall Space and Its Effect on Empirical Evaluation.
Date
2012-12
Journal Title
Journal ISSN
Volume Title
Repository Usage Stats
views
downloads
Attention Stats
Abstract
Precision-recall (PR) curves and the areas under them are widely used to summarize machine learning results, especially for data sets exhibiting class skew. They are often used analogously to ROC curves and the area under ROC curves. It is known that PR curves vary as class skew changes. What was not recognized before this paper is that there is a region of PR space that is completely unachievable, and the size of this region depends only on the skew. This paper precisely characterizes the size of that region and discusses its implications for empirical evaluation methodology in machine learning.
Type
Department
Description
Provenance
Subjects
Citation
Permalink
Collections
Scholars@Duke
David Page
David Page, PhD, serves as chair of the Department of Biostatistics and Bioinformatics and professor of biostatistics and bioinformatics and computer science at Duke University. He joined Duke in 2019. Dr. Page works on algorithms for data mining and machine learning and their applications to biomedical data. His research focuses on machine learning methods for complex multi-relational data, such as electronic health records, high throughput genetic and molecular data, and irregular temporal data, and methods that find causal relationships and produce human-interpretable output.
Dr. Page co-leads the Duke Discovery AI initiative which unites computational scientists, biologists, and engineers across the university to advance integration of artificial intelligence with biological research at the molecular, cellular, and organism scales. Together they train the next generation of scientists in biological systems and computational methods.
During his 20 years at the University of Wisconsin-Madison, Dr. Page taught courses titled Advanced AI, Machine Learning, Bioinformatics, and Health Informatics, in addition to special topics courses on Statistical Relational Learning and Learning Biological Networks. Dr. Page was a Kellett and Vilas Distinguished Achievement Professor and was Director of the Informatics Core of the Carbone Cancer Center. He served on scientific advisory and leadership committees for the Observational Medical Outcomes Partnership (OMOP), the International Warfarin Pharmacogenetics Consortium (IWPC), the Wisconsin Genomics Initiative, and UW-Madison's Institute for Clinical & Translational Science. Dr. Page received his PhD in computer science from the University of Illinois at Urbana-Champaign, where his dissertation focused on theoretical aspects of machine learning. He first became involved in biomedical applications of machine learning during his post-doc working with Dr. Stephen Muggleton at Oxford University.
Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.