PatchLens: Deep Learning and Statistics-based Pathological Anomaly Detection In Gastrointestinal Whole Slide Images

Abstract

Histopathology whole slide images (WSIs) examination serves as the gold standard for diagnosing gastrointestinal diseases, yet it remains as a cognitively demanding process for pathologists to navigate gigapixel images and identify subtle regions of morphological abnormalities. While recent advances in computational pathology have demonstrated the effectiveness of deep learning methods in WSI analysis, traditional supervised models often focus on slide-level classification and lack spatial interpretability to provide the fine-grained localization required for practical clinical workflows. Furthermore, existing reconstruction-based anomaly detection methods also struggle to model the complex, multi-subtype nature of pathological tissue and are frequently limited in providing interpretable results for reliable clinical decision support.In this paper, we propose PatchLens, a multi-decoder anomaly detection framework designed to identify abnormal tissue regions by modeling patch reconstruction behavior across multiple disease-conditioned manifolds. The study utilizes a dataset of 2,039 gastrointestinal WSIs from Duke Hospital across eight diagnostic categories including normal tissue to pathological conditions of different severity. In the proposed pipeline, WSIs are first tiled into non-overlapping patches and filtered for artifacts using an unsupervised clustering-based procedure. The remaining tissue patches are encoded into high-dimensional embeddings using the UNI2-h pathology foundation model. To improve robustness against acquisition variability, a slide-invariant generalist encoder is trained using a Gradient Reversal Layer (GRL) to suppress slide-specific signals and batch effects. Then, set of subtype-specific decoders is learned to reconstruct these embeddings to model distinct pathological contexts. Reconstruction errors are subsequently interpreted relative to empirical reference distributions derived from a held-out validation set to enable statistically grounded evaluation of patch-level deviations. Final anomaly scores are computed by aggregating these statistical signals using Stouffer’s Z-score method, which quantifies a patch’s deviation from normal tissue patterns across multiple disease contexts.Experimental results demonstrate that PatchLens effectively identifies clinically meaningful morphological deviations. The proposed anomaly score consistently outperforms simple reconstruction baselines and achieves performance comparable to supervised Multiple Instance Learning (MIL) baselines across multiple evaluation cohorts. Furthermore, through qualitative analysis using an interactive front-end visualization tool, we demonstrate that our system successfully highlights pathological structures such as cancerous mucosa and intestinal metaplasia while maintaining low false-positive rates on normal tissue and non-pathological structures such as blood regions. These findings suggest that PatchLens provides a robust and interpretable framework for integrating anomaly-driven visualization into real-world pathology inspection workflows.