Integration of Hand-Crafted Features and Deep Learning for Imaging Application in Precision Medicine

Abstract

The advent in imaging techniques have revolutionized the field of precision medicine, enabling early warnings, precise diagnoses, and long-term monitoring of diseases. This dissertation explores the integration of hand-crafted features and deep learning techniques in two distinct imaging applications for precision medicine: stool analysis using images collected by a smart toilet, and vessel characterization using digital pathology images of renal pathology.For stool analysis, a computer vision-based approach was developed to provide clinically and physiologically relevant information on stool form and color, enhancing gastrointestinal health assessments. Due to the absence of existing datasets, a comprehensive stool image dataset was constructed, facilitating the development of classification approaches and standardized color annotation methods. We addressed three limitations associated with the application of computer-vision techniques to a smart toilet system: (i) uneven separability between different stool form categories; (ii) class imbalance in the dataset; (iii) limited computational resources available in the devices used for capturing stool images. Experimental results demonstrate the effectiveness in hierarchical convolutional neural network architectures for training a stool-form classifier and in perceptual color quantization coupled with machine-learning techniques to optimize the color feature space for the classification of stool color. To align our solution with practical applications, particularly within an IoT-based healthcare scenario, we further provide an edge-assisted architecture and investigate the trade-offs between edge processing and cloud computing for both functions.We further validated the feasibility and efficacy of automated stool form analysis using images collected by a smart toilet system, and developed a computational pipeline to accurately categorize stool forms based on the Bristol Stool Form Scale. Utilizing images collected from six healthy subjects by the smart toilet system, developed by our research group, our computational pipeline integrates an open-source, pre-trained deep learning model with zero-shot capabilities to accurately segment stool images. Following segmentation, we developed a suite of quantitative features to characterize stool consistency, including size, texture, and morphology. Our analysis revealed that multiple extracted features individually demonstrate clear separability between different stool form categories. We also investigated the use of machine learning techniques for stool form classification and developed a rule-based classifier that achieved high accuracy against expert annotations. This validation provides a solid foundation for the application of the smart toilet system in both clinical and research settings.For vessel characterization, we focus on the histologic parameters of arterio- and arteriolosclerosis, which involve the fibrous thickening of the intima and hypertrophy of the media in the arteries and arterioles of the kidney, a critical indicator of kidney disease progression. We developed a novel hand-crafted computer vision approach using 2D ray casting and radial sampling to systematically measure the thickness of the intima and media. This method represents both media and intima thickness as a function of spatially encoded polar coordinates along the entire arterial perimeter, allowing the application of signal-smoothing techniques to effectively mitigate the impact of variable arterial morphology caused during tissue harvesting and preparation. A numerical validation of our approach, based on simulated arterial shape distortion, demonstrated the feasibility of this technique. Experimental results on patient data revealed that pathomic features extracted from the measurements of intima and media thickness show clear separability between arterioles/arteries with different severities of arteriosclerosis, underscoring the potential of our approach to enhance the assessment of arteriosclerosis in clinical practice and improve diagnostic accuracy.Lastly, we investigated the clinical relevance of computationally derived attributes of arterioles/arteries using digital kidney biopsies of patients with focal segmental glomerulosclerosis and minimal change disease. We developed a computational approach to quantify vascular characteristics through image segmentation and pathomic feature extraction. Specifically, we segmented viable muscular vessels along with their intra-vascular compartments (media, intima, lumen), and categorized them into arterioles, interlobular arteries, and arcuate arteries, each visually scored for arteriosclerosis and hyalinosis severity. We proposed a suite of pathomic features based on intra-vascular area and thickness to characterize arteries and arterioles. Our findings revealed a statistically significant association between pathomic features and visual scores of arteriosclerosis and hyalinosis at the vessel level. Furthermore, at the patient level, we found a significant association between proposed pathomic features, particularly within arterioles, and disease progression. Additionally, we identified individual pathomic features that contribute meaningfully to clinical outcomes, highlighting their potential to enhance diagnostic and prognostic capabilities in renal pathology.This dissertation demonstrates the effectiveness of utilizing computational image analysis techniques, specifically the integration of hand-crafted features and deep learning, across two distinct domains of precision medicine: stool analysis using images collected by a Smart Toilet, and vessel characterization using digital pathology images of renal pathology. These advancements are expected to benefit additional domains of precision medicine, extending their impact beyond the specific applications explored here.