Deep Learning Image Enhancement for Point of Care Ultrasound

Abstract

Deep Learning has opened several new applications in medical imaging. Specifically, Convolutional Neural Networks (CNNs) have been a widely successful backbone to image noise reduction, super resolution, and are able to match human-level per- formance in certain classification tasks. CNNs enable several novel applications in medical imaging where they can boost diagnostic signals and serve as predictive models which do not fatigue and may help reduce the amount of human training required to derive clinical value. In this work, we apply deep learning image enhancement techniques to the field of ultrasound. We investigate clinical screening applications it enables. Finally, we cover our contributions to open source software which aims to better translate these and future medical imaging research. In chapter 2, we apply a generative-adversarial method to mimick post-processing in modern clinical ultrasound scanners. Image post-processing is typically a separate step after beamforming and accounts for a significant amount of speckle noise reduction as well as increase in anatomical conspicuity. Our post-processing method achieves a 0.940 ̆0.018 structural similarity index measurement (SSIM) compared to clinical-grade post-processing on a 400 cine-loop test set and 0.937 ̆0.025 SSIM on a prospectively acquired dataset. This work enables apples-to-apples reader studies of the status quo versus novel imaging methods which are challenged by medical providers’ strong preference for commercially post-processed images. While the method presented in chapter 2 works for large conventional GPU hardware and bulky conventional ultrasound, real-time processing is not possible in more resource constrained mobile hardware. This is particularly significant given the rise of pocket-size point-of-care ultrasound. In chapter 3, we explore the use of Tensor Processing Units (TPUs) which aim to perform common deep learning operations using only 2W of power with a form factor of 10 mm x 15 mm. Using TPU hardware, we are able to achieve a 7.5x - 10x runtime increase compared to smartphone CPU. This enables real-time processing at 45 FPS for a 736x160 harmonic image. Addition- ally, the approximations used by TPU hardware result in outputs nearly identical to conventional GPU with an structural similarity index measurement (SSIM) of 0.98 ̆0.001 and a mean squared error (MSE) of 0.0001 ̆0.0 over our test set. This chapter highlights how to achieve real-time deep learning based post-processing. Ultimately, ultrasound image enhancement is meant to make anatomical struc- tures more clear with the source of contrast as the underlying tissue echogenicity map. Unfortunately, the echogenicity map is not known in vivo. In chapter 4, we use fullwave to simulate ultrasound data on a synthetic dataset modeling speckle and aberration noise. We use CNNs to recover the known raw echogenicity. With- out aberration, we can recover a remarkable amount of detail present in the original echogenicity map. However, with aberration clutter present, we are far from match- ing image quality without aberration. The tools and datasets produced can be used for future work exploring the use of simulation training data. In chapters 5 and 6, we cover two clinical screening applications in tympanom- etry and vascular ultrasound. We explore the use of aleatoric uncertainty modeling methods to detect abnormal 1-D tympanometry tracings. We achieve an AUC of 0.987 and are able to predict high variance when tracings are noisy. We also ex- plore multiple-instance-learning in the case of predicting spontaneous echo contrast (SEC), a finding present in patients at risk for blood clot. In this work, we have a weakly labeled dataset. Using attention based MIL, we can achieve an AUC of 0.74 to detect SEC. In chapter 7, we cover modern medical imaging visualization tools in the browser. We highlight our software contributions to the Open Health Imaging Foundation, Cornerstone-3D, and the Visualization Toolkit, VTK.js. We develop an API to en- able touch control for iPad and mobile phones. We enable a method for approximat- ing sigmoidal look up tables. Finally, we enable the support of progressively loading 16-bit volume textures for more memory efficient medical data display. These tools were built with the goal of help translational medical imaging research.