Browsing by Author "Wang, Chunhao"

Purpose: To compare radiomics and deep-learning (DL) methods for predicting NSCLC surgical treatment failure. Methods: A cohort of 83 patients undergoing lobectomy or wedge resection for early-stage NSCLC from our institution was studied. There were 7 local failures and 16 non-local failures (regional and/or distant). Gross tumor volumes (GTV) were contoured on pre-surgery CT datasets after 1mm3 isotropic resolution resampling. For the radiomics analysis, 92 radiomics features were extracted from the GTV and z-score normalizations were performed. The multivariate association between the extracted features and clinical endpoints were investigated using a random forest model following 70%-30% training-test split. For the DL analysis, both 2D and 3D model designs were executed using two different deep neural networks as transfer learning problems: in 2D-based design, 8x8cm2 axial fields-of-view(FOVs) centered within the GTV were adopted for VGG-16 training; in 3D-based design, 8x8x8 cm3 FOVs centered within the GTV were adopted for U-Net’s encoder path training. In both designs, data augmentation (rotation, translation, flip, noise) was included to overcome potential training convergence problems due to the imbalanced dataset, and the same 70%-30% training-test split was used. The performances of the 3 models (Radiomics, 2D-DL, 3D-DL) were tested to predict outcomes including local failure, non-local failure, and disease-free survival. Sensitivity/specificity/accuracy/ROC results were obtained from their 20 trained versions. Results: The radiomics models showed limited performances in all three outcome prediction tasks. The 2D-DL design showed significant improvement compared to the radiomics results in predicting local failure (ROC AUC = 0.546±0.056). The 3D-DL design achieved the best performance for all three outcomes (local failure ROC AUC = 0.768 ± 0.051, non-local failure ROC AUC = 0.683±0.027, disease-free ROC AUC = 0.694±0.042) with statistically significant improvements from radiomics/2D-DL results. Conclusions: 3D-DL execution outperformed the 2D-DL in predicting clinical outcomes after surgery for early-stage NSCLC. By contrast, classic radiomics approach did not achieve satisfactory results.

Brain metastases are a common complication of many types of cancer, including lung, breast, and melanoma. Approximately 30-40% of patients develop brain metastases that originate from primary systemic tumors during the course of cancer treatment. One treatment method is a LINAC-based single-isocenter multiple-target (SIMT) stereotactic radiosurgery (SRS). High plan quality has been one of the important goals in radiotherapy treatment planning. Generation of a high quality SRS treatment plan, particularly a SIMT plan, usually requires planners’ extensive planning experience, multiple runs of planning and trial-and-error, and frequent communication among planners, physicians and other radiation oncology team members. In clinical practice with potentially limited resources, SIMT SRS planning could be time-consuming and may have large variations in plan dosimetric quality. Therefore, an estimation of achievable dosimetric outcome can help reduce plan quality variation and improve planning efficiency. Assuming 20Gy in a single fraction of treatment, the volume of normal brain tissue receiving 10Gy (V50%), 12Gy (V60%), and 13Gy (V66.7%) are known predictors of brain tissue toxicity, or radionecrosis. We developed deep learning networks for the prediction of V50%, V60%, and V66.7% based on each patient’s target delineation. A prediction of achievable V10Gy, V12Gy, and V13Gy (assuming 20Gy x 1fx) can assist physicians in the determination of fractionation schemes (i.e., single fx vs. multiple fx). Such predictions can be used as guidelines for planners to generate a SIMT plan more rapidly with reduced dosimetric variability. A key technical innovation of this work is the spherical projection design: by projecting target distribution on a spherical surface, the target distribution in 3D is collapsed to a polar-azimuthal angular distribution map. This transformation enables a dimensional reduction for deep learning input without losing volumetric information. Our results indicate promising potential but there is a need for further work to improve the accuracy of our predictions.

Purpose: To predict post-radiotherapy overall survival group of NSCLC patients based on clinical information and radiomics analysis of simulation CT. Materials/Methods: A total of 258 non-adenocarcinoma patients who received radical radiotherapy or chemo-radiation were studied: 45/50/163 patients were identified as short(0-6mos)/mid(6-12mos)/long(12+mos) survival groups, respectively. For each patient, we first extracted 76 radiomics features within the gross tumor volume(GTV) identified in the simulation CT; these features were combined with patient clinical information (age, overall stage, and GTV volume) as a patient-specific feature vector, which was utilized by a 2-step machine learning model for survival group prediction. This model first identifies patients with long survival prediction via a supervised binary classifier; for those with otherwise prediction, a 2nd classifier further generates short/mid survival prediction. Two machine learning classifiers, explainable boosting machine(EBM) and balanced random forest(BRF), were interrogated as a comparison study. During the model training, all patients were divided into training/test sets by an 8:2 ratio, and 100-fold random sampling were applied to the training set with a 7:1 validation ratio. Model performances were evaluated by the sensitivity, accuracy, and ROC results. Results: The model with EBM demonstrated an overall ROC AUC (0.58±0.04) with limited sensitivities in short (0.02±0.04) and mid group (0.11±0.08) predictions due to imbalanced data sample distribution. In contrast, the model with BRF improved short/mid group sensitivities to 0.32±0.11/0.29±0.16, respectively, but the improvement of ROC AUC (0.60±0.04) is limited. Nevertheless, both EBM (0.46±0.04) and BRF (0.57±0.04) approaches achieved limited overall accuracy; a noticeable overlap was found in their feature lists with top 10 feature weight rankings. Conclusion: The proposed two-step machine learning model with BRF classifier possesses a better performance than the one with EBM classifier in the post-radiotherapy survival group prediction of NSCLC. Future works, preferably in the joint use of deep learning, are in demand to further improve the prediction results.

AbstractPurpose: To develop a deep ensemble learning model with a radiomics spatial encoding execution for improved glioma segmentation accuracy using multi-parametric MRI (mp-MRI). Materials/Methods: This radiomics-incorporated deep ensemble learning model was developed using 369 glioma patients with a 4-modality mp-MRI protocol: T1, contrast-enhanced T1 (T1-Ce), T2, and FLAIR. In each modality volume, a 3D sliding kernel was implemented across the brain to capture image heterogeneity: fifty-six radiomic features were extracted within the kernel, resulting in a 4th order tensor. Each radiomic feature can then be encoded as a 3D image volume, namely a radiomic feature map (RFM). For each patient, all RFMs extracted from all 4 modalities were processed by the Principal Component Analysis (PCA) for dimension reduction, and the first 4 principal components (PCs) were selected. Next, four deep neural networks following the U-net’s architecture were trained for the segmenting of a region-of-interest (ROI): each network utilizes the mp-MRI and 1 of the 4 PCs as a 5-channel input for 2D execution. Last, the 4 softmax probability results given by the U-net ensemble were superimposed and binarized by Otsu’s method as the segmentation result. Three deep ensemble models were trained to segment enhancing tumor (ET), tumor core (TC), and whole tumor (WT), respectively. Segmentation results given by the proposed ensemble were compared to the mp-MRI-only U-net results. Results: All 3 radiomics-incorporated deep learning ensemble models were successfully implemented: Compared to mp-MRI-only U-net results, the dice coefficients of ET (0.777→0.817), TC (0.742→0.757), and WT (0.823→0.854) demonstrated improvements. Accuracy, sensitivity, and specificity results demonstrated the same patterns. Conclusion: The adopted radiomics spatial encoding execution enriches the image heterogeneity information that leads to the successful demonstration of the proposed neural network ensemble design, which offers a new tool for mp-MRI-based medical image segmentation.

Purpose: The purpose of this study is to develop and validate two deep learning (DL) models for the management of brain metastasis (BM) patients treated with stereotactic radiosurgery (SRS). The first model is a radiomics-integrated deep learning (RIDL) model, which aims to distinguish between radionecrosis and tumor recurrence in patients with post-SRS radiographic progression. The second model, a novel dose-incorporated deep ensemble learning (DEL) model, aims to accurately predict local failure outcomes in brain metastasis patients following SRS.Materials/Methods: A total of 51 patients with post-SRS radiographic progression (37 radionecrosis, 14 recurrence) and 114 BMs (including 26 BMs that developed biopsy-confirmed local failure post-SRS) from 85 patients were included in this study. For the first aim, a radiomics-integrated deep learning (RIDL) model was developed using three steps: 1) 184 radiomics features (RFs) were extracted from the SRS planning target volume (PTV) and 60% isodose volume (V60%); 2) a deep neural network (DNN) mimicking the encoding path of U-net was trained for radionecrosis or recurrence prediction using the 3D MR volume. Prior to the binary prediction output, latent variables in the DNN were extracted as 512 deep features (DFs); and 3) all extracted features were synthesized as a multi-dimensional input for support vector machine (SVM) execution. Key features with higher linear kernel weighting factors were identified by clustering analysis and were utilized by SVM to predict radionecrosis or recurrence. During model training, 50 model versions were acquired with random validation sample assignments following an 8:2 training/test ratio, and sensitivity, specificity, accuracy, and ROC were evaluated and compared with results from a radiomics-only and a DNN-only prediction model. For the second aim, a novel dose-incorporated deep ensemble learning (DEL) model was developed. The DEL design included four VGG-19 deep encoder networks, and each sub-network utilized a different variable type as input for BM outcome prediction. The DEL's outcome was synthesized from the four sub-network results via logistic regression. For each BM, four variables were obtained, including three with different curvatures during spherical projection and one with the original planar images. The proposed DEL model was developed using an 8:2 ratio for training/test assignment, and 10 model versions were acquired with random validation sample assignments. The DEL model performance was compared based on ROC analysis to a single VGG-19 encoder and to DEL models with the same projection designs, which used T1-CE MRI as the only input. Results: The RIDL model demonstrated superior performance compared to radiomics-based and DNN-only prediction models for distinguishing radionecrosis from tumor recurrence in brain metastasis patients with post-SRS radiographic progression. The RIDL model achieved the best prediction accuracy (0.643±0.059) and sensitivity (0.650±0.122) results with 32 identified key features (3 RFs+29 DFs), and it also demonstrated superior ROC results (AUC=0.688±0.035). In addition, for patients with NSCLC primary disease, the RF joint energy extracted from V60% and one DF correlated with ALK/EGFR mutations, respectively. Moreover, the DEL model achieved an excellent ROC AUC=0.84±0.03 with high sensitivity (0.78±0.08), specificity (0.81±0.09), and accuracy (0.80±0.06) results. This outperformed the MRI-only single VGG-19 encoder (sensitivity:0.35±0.01, AUC:0.64±0.08) and the MRI-only DEL (sensitivity:0.60±0.09, AUC:0.68±0.06) models. Conclusions: The RIDL model successfully differentiates brain metastasis radionecrosis from recurrence using a single post-SRS MR scan. Integration of clinical and treatment-related features is warranted to develop a comprehensive clinico-radiomic model. Additionally, the dose-incorporated DEL model design demonstrated robust and promising performance. It could potentially improve other radiotherapy outcome models and warrant further evaluation.

Purpose: To investigate the feasibility of integrate radiomics and deep learning in computer-aided medical imaging analysis Methods: Two different approaches were investigated to integrate radiomics and deep leaning on two independent tasks respectively. In the first approach, a 2D sliding kernel was implemented to map the impulse response of radiomic features throughout the entire chest X-ray image; thus, each feature is rendered as a 2D map in the same dimension as the X-ray image. Based on each of the three investigated deep neural network architectures, including VGG-16, VGG-19, and DenseNet-121, a pilot model was trained using X-ray images only. Subsequently, 2 radiomic feature maps (RFMs) were selected based on cross-correlation analysis in reference to the pilot model saliency map results. The radiomics-boosted model was then trained based on the same deep neural network architecture using X-ray images plus the selected RFMs as input. The proposed radiomics-boosted design was developed using 812 chest X-ray images with 262/288/262 COVID-19/Non-COVID-19 pneumonia/healthy cases, and 649/163 cases were assigned as training-validation/independent test sets. For each model, 50 runs were trained with random assignments of training/validation cases following the 7:1 ratio in the training-validation set. Sensitivity, specificity, accuracy, and ROC curves together with Area-Under-the-Curve (AUC) from all three deep neural network architectures were evaluated. In the second approach, a cohort of 235 GBM patients with complete surgical resection was divided into short-term/long-term survival groups with 1-yr survival time threshold. Each patient received a pre-surgery multi-parametric MRI exam with 4 scans: T1, contrast-enhanced T1 (T1ce), T2, and FLAIR. Three tumor subregions were segmented by neuroradiologists, and the whole dataset was divided into training, validation, and test groups following a 7:1:2 ratio. The developed model comprises three data source branches: in the 1st radiomics branch, 456 radiomics features (RF) were calculated from the three tumor subregions of each patient’s MR images; in the 2nd deep learning branch, an encoding neural network architecture was trained for survival group prediction using each single MR modality, and high-dimensional parameters from the last two network layers were extracted as deep features (DF). The extracted radiomics features and deep features were processed by a feature selection procedure to reduce the dimension size of each feature space. In the 3rd branch, patient-specific clinical features (PSCF), including patient age and three tumor subregions volumes, were collected from the dataset. Finally, data sources from all three branches were fused as an integrated input for a supporting vector machine (SVM) execution for survival group prediction. Different strategies of model design were investigated in comparison studies, including 1) 2D/3D-based image analysis, 2) different radiomics feature space dimension reduction methods, and 3) different data source combinations in SVM input design. Results: In the first approach, all three investigated deep neural network architectures demonstrated improved sensitivity, specificity, accuracy, and ROC AUC results in COVID-19 and healthy individual classifications. VGG-16 showed the largest improvement in COVID-19 classification ROC (AUC from 0.963 to 0.993), and DenseNet-121 showed the largest improvement in healthy individual classification ROC (AUC from 0.962 to 0.989). The reduced variations suggested improved robustness of the model to data partition. For the challenging Non-COVID-19 pneumonia classification task, radiomics-boosted implementation of VGG-16 (AUC from 0.918 to 0.969) and VGG-19 (AUC from 0.964 to 0.970) improved ROC results, while DenseNet-121 showed a slight yet insignificant ROC performance reduction (AUC from 0.963 to 0.949). The achieved highest accuracy of COVID-19/Non-COVID-19 pneumonia/healthy individual classifications were 0.973 (VGG-19)/0.936 (VGG-19)/ 0.933 (VGG-16), respectively. In the second approach, the model achieved 0.638 prediction accuracy in the test set when using patient-specific clinical features only, which was higher than the results using radiomics features/deep features as sole input of SVM in both 2D and 3D based analysis. The inclusion of radiomics features or deep features with patient-specific clinical features improved accuracy results in 3D analysis. The most accurate models in 2D/3D analysis reached the highest accuracy of 0.745 with different combinations of dissimilarity-selected radiomics features, deep features, and patient-specific clinical features, and the corresponding ROC area-under-curve (AUC) results were 0.69 (2D) and 0.71 (3D), respectively.

Conclusions: The integration of radiomic analysis in deep learning model design improved the performance and robustness computer-aided diagnosis and outcome predication, which holds great potential for clinical applications and provides a radiomics perspective for deep learning interpretation.

Abstract

The goal of modern radiotherapy is to precisely deliver a prescribed radiation dose to delineated target volumes that contain a significant amount of tumor cells while sparing the surrounding healthy tissues/organs. Precise delineation of treatment and avoidance volumes is the key for the precision radiation therapy. In recent years, considerable clinical and research efforts have been devoted to integrate MRI into radiotherapy workflow motivated by the superior soft tissue contrast and functional imaging possibility. Dynamic contrast-enhanced MRI (DCE-MRI) is a noninvasive technique that measures properties of tissue microvasculature. Its sensitivity to radiation-induced vascular pharmacokinetic (PK) changes has been preliminary demonstrated. In spite of its great potential, two major challenges have limited DCE-MRI’s clinical application in radiotherapy assessment: the technical limitations of accurate DCE-MRI imaging implementation and the need of novel DCE-MRI data analysis methods for richer functional heterogeneity information.

This study aims at improving current DCE-MRI techniques and developing new DCE-MRI analysis methods for particular radiotherapy assessment. Thus, the study is naturally divided into two parts. The first part focuses on DCE-MRI temporal resolution as one of the key DCE-MRI technical factors, and some improvements regarding DCE-MRI temporal resolution are proposed; the second part explores the potential value of image heterogeneity analysis and multiple PK model combination for therapeutic response assessment, and several novel DCE-MRI data analysis methods are developed.

I. Improvement of DCE-MRI temporal resolution. First, the feasibility of improving DCE-MRI temporal resolution via image undersampling was studied. Specifically, a novel MR image iterative reconstruction algorithm was studied for DCE-MRI reconstruction. This algorithm was built on the recently developed compress sensing (CS) theory. By utilizing a limited k-space acquisition with shorter imaging time, images can be reconstructed in an iterative fashion under the regularization of a newly proposed total generalized variation (TGV) penalty term. In the retrospective study of brain radiosurgery patient DCE-MRI scans under IRB-approval, the clinically obtained image data was selected as reference data, and the simulated accelerated k-space acquisition was generated via undersampling the reference image full k-space with designed sampling grids. Two undersampling strategies were proposed: 1) a radial multi-ray grid with a special angular distribution was adopted to sample each slice of the full k-space; 2) a Cartesian random sampling grid series with spatiotemporal constraints from adjacent frames was adopted to sample the dynamic k-space series at a slice location. Two sets of PK parameters’ maps were generated from the undersampled data and from the fully-sampled data, respectively. Multiple quantitative measurements and statistical studies were performed to evaluate the accuracy of PK maps generated from the undersampled data in reference to the PK maps generated from the fully-sampled data. Results showed that at a simulated acceleration factor of four, PK maps could be faithfully calculated from the DCE images that were reconstructed using undersampled data, and no statistically significant differences were found between the regional PK mean values from undersampled and fully-sampled data sets. DCE-MRI acceleration using the investigated image reconstruction method has been suggested as feasible and promising.

Second, for high temporal resolution DCE-MRI, a new PK model fitting method was developed to solve PK parameters for better calculation accuracy and efficiency. This method is based on a derivative-based deformation of the commonly used Tofts PK model, which is presented as an integrative expression. This method also includes an advanced Kolmogorov-Zurbenko (KZ) filter to remove the potential noise effect in data and solve the PK parameter as a linear problem in matrix format. In the computer simulation study, PK parameters representing typical intracranial values were selected as references to simulated DCE-MRI data for different temporal resolution and different data noise level. Results showed that at both high temporal resolutions (<1s) and clinically feasible temporal resolution (~5s), this new method was able to calculate PK parameters more accurate than the current calculation methods at clinically relevant noise levels; at high temporal resolutions, the calculation efficiency of this new method was superior to current methods in an order of 102. In a retrospective of clinical brain DCE-MRI scans, the PK maps derived from the proposed method were comparable with the results from current methods. Based on these results, it can be concluded that this new method can be used for accurate and efficient PK model fitting for high temporal resolution DCE-MRI.

II. Development of DCE-MRI analysis methods for therapeutic response assessment. This part aims at methodology developments in two approaches. The first one is to develop model-free analysis method for DCE-MRI functional heterogeneity evaluation. This approach is inspired by the rationale that radiotherapy-induced functional change could be heterogeneous across the treatment area. The first effort was spent on a translational investigation of classic fractal dimension theory for DCE-MRI therapeutic response assessment. In a small-animal anti-angiogenesis drug therapy experiment, the randomly assigned treatment/control groups received multiple fraction treatments with one pre-treatment and multiple post-treatment high spatiotemporal DCE-MRI scans. In the post-treatment scan two weeks after the start, the investigated Rényi dimensions of the classic PK rate constant map demonstrated significant differences between the treatment and the control groups; when Rényi dimensions were adopted for treatment/control group classification, the achieved accuracy was higher than the accuracy from using conventional PK parameter statistics. Following this pilot work, two novel texture analysis methods were proposed. First, a new technique called Gray Level Local Power Matrix (GLLPM) was developed. It intends to solve the lack of temporal information and poor calculation efficiency of the commonly used Gray Level Co-Occurrence Matrix (GLCOM) techniques. In the same small animal experiment, the dynamic curves of Haralick texture features derived from the GLLPM had an overall better performance than the corresponding curves derived from current GLCOM techniques in treatment/control separation and classification. The second developed method is dynamic Fractal Signature Dissimilarity (FSD) analysis. Inspired by the classic fractal dimension theory, this method measures the dynamics of tumor heterogeneity during the contrast agent uptake in a quantitative fashion on DCE images. In the small animal experiment mentioned before, the selected parameters from dynamic FSD analysis showed significant differences between treatment/control groups as early as after 1 treatment fraction; in contrast, metrics from conventional PK analysis showed significant differences only after 3 treatment fractions. When using dynamic FSD parameters, the treatment/control group classification after 1st treatment fraction was improved than using conventional PK statistics. These results suggest the promising application of this novel method for capturing early therapeutic response.

The second approach of developing novel DCE-MRI methods is to combine PK information from multiple PK models. Currently, the classic Tofts model or its alternative version has been widely adopted for DCE-MRI analysis as a gold-standard approach for therapeutic response assessment. Previously, a shutter-speed (SS) model was proposed to incorporate transcytolemmal water exchange effect into contrast agent concentration quantification. In spite of richer biological assumption, its application in therapeutic response assessment is limited. It might be intriguing to combine the information from the SS model and from the classic Tofts model to explore potential new biological information for treatment assessment. The feasibility of this idea was investigated in the same small animal experiment. The SS model was compared against the Tofts model for therapeutic response assessment using PK parameter regional mean value comparison. Based on the modeled transcytolemmal water exchange rate, a biological subvolume was proposed and was automatically identified using histogram analysis. Within the biological subvolume, the PK rate constant derived from the SS model were proved to be superior to the one from Tofts model in treatment/control separation and classification. Furthermore, novel biomarkers were designed to integrate PK rate constants from these two models. When being evaluated in the biological subvolume, this biomarker was able to reflect significant treatment/control difference in both post-treatment evaluation. These results confirm the potential value of SS model as well as its combination with Tofts model for therapeutic response assessment.

In summary, this study addressed two problems of DCE-MRI application in radiotherapy assessment. In the first part, a method of accelerating DCE-MRI acquisition for better temporal resolution was investigated, and a novel PK model fitting algorithm was proposed for high temporal resolution DCE-MRI. In the second part, two model-free texture analysis methods and a multiple-model analysis method were developed for DCE-MRI therapeutic response assessment. The presented works could benefit the future DCE-MRI routine clinical application in radiotherapy assessment.

Meningiomas are the most common primary brain tumors and often times treated throughradiation therapy. With the surge of interest in deep neural network (DNN) model applications, image segmentation of meningioma radiotherapy target, gross tumor volume (GTV), can improve clinical outcomes and clinic efficiency. A major limitation of DNN applications is the lack of a standardized methodology for quantification of uncertainty. DNNs are prone to making unexpected errors, hindering its safe transition into clinical applications. This work investigates the use of a spherical projection-based U-Net (SPU-Net) segmentation model to improve meningioma segmentation performance and allow for uncertainty quantification. As an equivalence of nonlinear image transform, spherical projection enhances locoregional details while maintaining the global field of view. By employing multiple projection centers, SPU-Net generates various GTV segmentation predictions, with the variance indicating the model's uncertainty. This uncertainty is quantified on a pixel-wise basis using entropy calculations and aggregated through Otsu’s method for final segmentation. The SPU-Net model surpassed the traditional U-Net in sensitivity (0.758 vs. 0.746), Dice coefficient (0.760 vs. 0.742), and reduced Hausdorff distance (2.682 cm vs 2.912 cm). Uncertainty mapping revealed low uncertainty in accurate segments (e.g., within GTV or healthy tissue) and higher uncertainty in problematic areas (e.g., GTV boundaries, dural tail), providing valuable insights for potential manual corrections. SPU-Net enhances the performance of MR-based meningioma GTV segmentation and provides uncertainty quantification. This advancement is particularly valuable given the complex extra-axial nature of meningiomas and their involvement with dural tissue, offering a significant improvement over traditional segmentation approaches.