Automatic identification of variables in epidemiological datasets using logic regression.

Thumbnail Image
View / Download1.06 MB

Date

2017-04

Authors

Journal Title

Journal ISSN

Volume Title

Repository Usage Stats

0
views
6
downloads

Citation Stats

Abstract

Background

For an individual participant data (IPD) meta-analysis, multiple datasets must be transformed in a consistent format, e.g. using uniform variable names. When large numbers of datasets have to be processed, this can be a time-consuming and error-prone task. Automated or semi-automated identification of variables can help to reduce the workload and improve the data quality. For semi-automation high sensitivity in the recognition of matching variables is particularly important, because it allows creating software which for a target variable presents a choice of source variables, from which a user can choose the matching one, with only low risk of having missed a correct source variable.

Methods

For each variable in a set of target variables, a number of simple rules were manually created. With logic regression, an optimal Boolean combination of these rules was searched for every target variable, using a random subset of a large database of epidemiological and clinical cohort data (construction subset). In a second subset of this database (validation subset), this optimal combination rules were validated.

Results

In the construction sample, 41 target variables were allocated on average with a positive predictive value (PPV) of 34%, and a negative predictive value (NPV) of 95%. In the validation sample, PPV was 33%, whereas NPV remained at 94%. In the construction sample, PPV was 50% or less in 63% of all variables, in the validation sample in 71% of all variables.

Conclusions

We demonstrated that the application of logic regression in a complex data management task in large epidemiological IPD meta-analyses is feasible. However, the performance of the algorithm is poor, which may require backup strategies.

Type

Journal article

Department

Description

Provenance

Subjects

PROG-IMT study group, Humans, Carotid Artery Diseases, Prognosis, Logistic Models, Predictive Value of Tests, Epidemiologic Factors, Algorithms, Databases, Factual, Medical Informatics Applications, Meta-Analysis as Topic, Data Mining, Carotid Intima-Media Thickness

Citation

Permalink

https://hdl.handle.net/10161/31168

Rights

https://creativecommons.org/licenses/by-nc/4.0

Published Version (Please cite this version)

10.1186/s12911-017-0429-1

Publication Info

Lorenz, Matthias W, Negin Ashtiani Abdi, Frank Scheckenbach, Anja Pflug, Alpaslan Bülbül, Alberico L Catapano, Stefan Agewall, Marat Ezhov, et al. (2017). Automatic identification of variables in epidemiological datasets using logic regression. BMC medical informatics and decision making, 17(1). p. 40. 10.1186/s12911-017-0429-1 Retrieved from https://hdl.handle.net/10161/31168.

This is constructed from limited available data and may be imprecise. To cite this article, please review & use the official citation provided by the journal.

Collections

Scholarly Articles
Full item page

Unless otherwise indicated, scholarly articles published by Duke faculty members are made available here with a CC-BY-NC (Creative Commons Attribution Non-Commercial) license, as enabled by the Duke Open Access Policy. If you wish to use the materials in ways not already permitted under CC-BY-NC, please consult the copyright owner. Other materials are made available here through the author’s grant of a non-exclusive license to make their work openly accessible.