Denoise high dimensional dataset with complicated noise and its clinical applications

Abstract

We present a novel algorithm, called eOptShrink, for denoising matrices in the presence of high-dimensional, colored, and dependent noise with a separable covariance structure. Our approach is data-driven and does not require estimation of the covariance structure of the noise.The eOptShrink algorithm utilizes a new imputation and rank estimation technique to achieve optimal shrinkage. We study the asymptotic behavior of the singular values and singular vectors of the random matrix associated with the noisy data, including the sticking property of non-outlier singular values and the delocalization of non-outlier singular vectors with a convergence rate. These theoretical results provide guarantees for the imputation, rank estimation, and eOptShrink algorithm with a convergence rate.We apply eOptShrink to recover fetal electrocardiogram (ECG) for both the fetal heart rate analysis and morphological analysis of its waveform from two or three trans-abdominal maternal ECG channels. For the fetal heart rate analysis, the algorithm is evaluated on publicly available database, 2013 PhyioNet/Computing in Cardiology Challenge, set A CinC2013. For the morphological analysis, we analyze CinC2013 and another publicly available database, Non-Invasive Fetal ECG Arrhythmia Database nifeadb, and propose to simulate semi-real databases by mixing the MIT-BIH Normal Sinus Rhythm Database and MITDB Arrhythmia Database. For the fetal R peak detection, the proposed algorithm outperforms all algorithms under comparison. For the morphological analysis, the algorithm provides an encouraging result in recovery of the fetal ECG waveform, including PR, QT and ST intervals, even when the fetus has arrhythmia, both in real and simulated databases.