Browsing by Subject "Convolutional neural network"

Purpose: Single photon emission computed tomography (SPECT) is considered as a functional nuclear medicine imaging technique which is commonly used in the clinic. However, it suffers from low resolution and high noise because of the physical structure and photon scatter and attenuation. This research aims to develop a compact neural network reconstructing SPECT images from projection data, with better resolution and low noise. Methods and Materials: This research developed a MATLAB program to generate 2-D brain phantoms. We totally generated 20,000 2-D phantoms and corresponding projection data. Furthermore, those projection data were processed with Gaussian filter and Poisson noise to simulate the real clinical situation. And 16,000 of them were used to train the neural network, 2,000 for validation, and the final 2,000 for testing. To simulate the real clinical situation, there are five groups of projection data with decreasing acquisition views are used to train the network. Inspired by the SPECTnet, we used a two-step training strategy for network design. The full-size phantom images (128×128 pixels) were compressed into a vector (256×1) at first, then they were decompressed to full-size images again. This process was achieved by the AutoEncoder (AE) consisting of encoder and decoder. The compressed vector generated by the encoder works as targets in the second network, which map projection to compressed images. Then those compressed vectors corresponding to the projection were reconstructed to full-size images by the decoder. Results: A total of 10,000 testing dataset divided into 5 groups with 360 degrees, 180 degrees, 150 degrees, 120 degrees and 90 degrees acquisition, respectively, are generated by the developed neural network. Results were compared with those generated by conventional FBP methods. Compared with FBP algorithm, the neural network can provide reconstruction images with high resolution and low noise, even if under the limited-angles acquisitions. In addition, the new neural network had a better performance than SPECTnet. Conclusions: The network successfully reconstruct projection data to activity images. Especially for the groups whose view angles is less than 180 degrees, the reconstruction images by neural network have the same excellent quality as other images reconstructed by projection data over 360 degrees, even has a higher efficiency than the SPECTnet. Keywords: SPECT; SPECT image reconstruction; Deep learning; convolution neural network. Purpose: Single photon emission computed tomography (SPECT) is considered as a functional nuclear medicine imaging technique which is commonly used in the clinic. However, it suffers from low resolution and high noise because of the physical structure and photon scatter and attenuation. This research aims to develop a compact neural network reconstructing SPECT images from projection data, with better resolution and low noise. Methods and Materials: This research developed a MATLAB program to generate 2-D brain phantoms. We totally generated 20,000 2-D phantoms and corresponding projection data. Furthermore, those projection data were processed with Gaussian filter and Poisson noise to simulate the real clinical situation. And 16,000 of them were used to train the neural network, 2,000 for validation, and the final 2,000 for testing. To simulate the real clinical situation, there are five groups of projection data with decreasing acquisition views are used to train the network. Inspired by the SPECTnet, we used a two-step training strategy for network design. The full-size phantom images (128×128 pixels) were compressed into a vector (256×1) at first, then they were decompressed to full-size images again. This process was achieved by the AutoEncoder (AE) consisting of encoder and decoder. The compressed vector generated by the encoder works as targets in the second network, which map projection to compressed images. Then those compressed vectors corresponding to the projection were reconstructed to full-size images by the decoder. Results: A total of 10,000 testing dataset divided into 5 groups with 360 degrees, 180 degrees, 150 degrees, 120 degrees and 90 degrees acquisition, respectively, are generated by the developed neural network. Results were compared with those generated by conventional FBP methods. Compared with FBP algorithm, the neural network can provide reconstruction images with high resolution and low noise, even if under the limited-angles acquisitions. In addition, the new neural network had a better performance than SPECTnet. Conclusions: The network successfully reconstruct projection data to activity images. Especially for the groups whose view angles is less than 180 degrees, the reconstruction images by neural network have the same excellent quality as other images reconstructed by projection data over 360 degrees, even has a higher efficiency than the SPECTnet. Keywords: SPECT; SPECT image reconstruction; Deep learning; convolution neural network.

Purpose: Cone beam computed tomography (CBCT) plays an important role in image guidance for interventional radiology and radiation therapy by providing 3D volumetric images of the patient. However, CBCT suffers from relatively low image quality with severe image artifacts due to the nature of the image acquisition and reconstruction process. This work investigated the feasibility of using deep learning networks to substantially augment the image quality of CBCT by learning a direct mapping from the original CBCT images to their corresponding ground truth CT images. The possibility of using deep learning for scatter correction in CBCT projections was also investigated.

Methods: Two deep learning networks, i.e. a symmetric residual convolutional neural network (SR-CNN) and a U-net convolutional network, were trained to use the input CBCT images to produce high-quality CBCT images that match with the corresponding ground truth CT images. Both clinical and Monte Carlo simulated datasets were included for model training. In order to eliminate the misalignments between CBCT and the corresponding CT, rigid registration was applied to clinical database. The binary masks achieved by Otsu auto-thresholding method were applied to for Monte Carlo simulate data to avoid the negative impact of non-anatomical structures on images. After model training, a new set of CBCT images were fed into the trained network to obtain augmented CBCT images, and the performances were evaluated and compared both qualitatively and quantitatively. The augmented CBCT images were quantitatively compared to CT using the peak-signal-to-noise ratio (PSNR) and the structural similarity index measure (SSIM).

Regarding the study for using deep learning for the scatter correction in CBCT, the scatter signal for each projection was acquired by Monte Carlo simulation. U-net model was trained to predict the scatter signals based on the original CBCT projections. Then the predicted scatter components were subtracted from the original CBCT projections to obtain scatter-corrected projections. CBCT image reconstructed by the scatter-corrected projections were quantitatively compared with that reconstructed by original projections.

Results: The augmented CBCT images by both SR-CNN and U-net models showed substantial improvement in image quality. Compared to original CBCT, the augmented CBCT images also achieve much higher PSNR and SSIM in quantitative evaluation. U-net demonstrated better performance than SR-CNN in quantitative evaluation and computational speed for CBCT image quality augmentation.

With the scatter correction in CBCT projections predicted by U-net, the scatter-corrected CBCT images demonstrated substantial improvement of the image contrast and anatomical details compared to the original CBCT images.

Conclusion: The proposed deep learning models can effectively augment CBCT image quality by correcting artifacts and reducing scatter. Given their relatively fast computational speeds and great performance, they can potentially become valuable tools to substantially enhance the quality of CBCT to improve its precision for target localization and adaptive radiotherapy.

Lung cancer has been the most commonly occurring cancer (J. Ferlay, 2018), with the highest fatality rate worldwide. Lung cancer patients undergoing radiation therapy typically experience many side effects. In order to reduce the adverse effects, lung function (ventilation state)-guided radiation therapy has been highly recommended. “Functional Lung Avoidance Radiation Therapy” (FLA-RT) can selectively avoid high-dose irradiation to the well-functioning region of the lungs and reduce lung injury during RT (Azza Ahmed Khalil, 2021). FLA-RT, however, needs information on lung function for the treatment process. The conventional techniques that acquire lung function map (S. W. Harders, 2013) include 99mTc SPECT technique (Suga, 2002), 99mTc MRI technique (LindsayMathew, 2012), 68Ga PET technique (Jason Callahan, 2013). Nevertheless, these techniques have the following issues: high cost, labor-intensive in the preparation process, and low accessibility for the radiation oncology departments.This research is aimed to investigate whether the lung function images could be generated from routine planning CT images using CNN. This study will also develop an image segmentation method to automatically and accurately segment lung volume for the chest CT images. This study retrospectively analyzed 99mTc DTPA SPECT scans of 21 cases. These were open-source data from "VAMPIRE (Ventilation Archive for Medical Pulmonary Image Registration Evaluation)" established by John Kipritidis, Henry C. Woodruff, and Paul J. Keall from the Radiation Physics Laboratory of the University of Sydney in Australia. The sizes for CT images and the reference mask images were 512 ⅹ 512 matrices with the pixel size of 2.5 ⅹ 2.5 mm2 and 3 mm slice thickness. The SPECT images were reconstructed in 512 ⅹ 512 matrices with 2.5 ⅹ 2.5 ⅹ 2.5 mm3 voxel size. CT, reference mask, and SPECT images are all in ". mha" data format for each study case. This study contains two major components. First, a deep-learning model was developed to auto-segment the lung region from the CT images. Second, another deep-learning model was developed to use the segmented lung CT image as input to predict lung ventilation function map. In order to accomplish the first task of this study, we used the CT images as the “network input” and the reference mask images as the "network-output." We then trained them with a designated 2D U-shape backbone network and successfully generated the first model. For testing the model performance, Pixel Accuracy, Pixel Recall, Pixel Precision, and Intersection of Union (IOU) were used as assessment criteria to evaluate the quality of model-generated lung masks based on the ground truth masks. In order to accomplish the second task, we used the segmented lung CT images as the “network input” and SPECT images as the "network-output." We then trained them with another designated 3D U-shape backbone network and successfully generated the second model. For testing the performance of the second model, the correlation coefficient (Spearman's coefficient) (Piantadosi) was used as assessment criteria to evaluate the correlation between the model-generated lung function images and the ground truth SPECT images. In order to achieve the optimal outcome, this study applied parallel studies that compared the influence of different training strategies on the outcome (see Chapter 3.3.2). The different train strategies include two aspects for DL Model 1 and four aspects for DL Model 2. Training the designed network with three-channels data as input provided the best results for image segmentation. For test case 1, the Pixel Accuracy is 0.935±0.033, the Pixel recall is 0.942±0.029, Pixel Precision is 0.942±0.032, and IoU is 0.891±0.042. For test case 2, the Pixel Accuracy is 0.950±0.024, the Pixel recall is 0.961±0.015, Pixel Precision is 0.943±0.028, and IoU is 0.909±0.036. For “deriving lung function images,” training the designed network using the ground truth mask to segment the chest CT with [-1,1] normalization and 32 ⅹ 32 ⅹ 64 training patch size as inputs provided the best results. The Spearman's correlation coefficients for cases 1 and 2 got 0.8689±0.038 and 0.8716±0.036, respectively. The preliminary study using the designed U-shape backbone convolutional neural networks (CNNs) achieved satisfactory auto-segmentation results and derived promising results of the lung function map. It indicates the feasibility of directly deriving the lung ventilation state (SPECT-like images) from CT images. The CNN-derived “SPECT-like” lung functional images might be used to reference FLA-RT.

Artificial intelligence (AI) reshapes computational imaging systems. Deep neural networks (DNN) not only show superior reconstruction performance over conventional ones handling the same sampling systems, these new reconstruction algorithms also call for new sampling strategies. In this dissertation, we study how DNN reconstruction algorithms and sampling strategy can be jointly designed to boost the system performance.

First, two DNNs for sensor fusion tasks based on convolutional neural networks (CNN) and transformers are proposed. They are able to fuse frames with different resolution, different wave band, or different temporal window. The amount of frames can also vary, showing great flexibility and scalability. A reasonable computational load is achieved by a proper receptive field design balancing the flexibility and complexity. Visual pleasing reconstruction results are achieved.

Then we demonstrate how DNN reconstruction algorithms favor certain sampling strategy over another, with snapshot compressive imaging (SCI) task as an example. Using synthetic datasets, we compare quasi-random coded sampling and multi-aperture multi-scale manifold sampling under DNN reconstruction. The latter sampling strategy requires much simpler physical setup, yet gives comparable, if not better, reconstruction image quality.

At the end, we design and build a multifocal array camera fitting the DNN reconstruction. With commercial on-the-shelf cameras and lenses, the array camera achieves a nearly 70 degree field of view (FoV), a 0.1m - 17.1m depth of field (DoF), and the ability to resolve objects with 2mm granularity. One final output image contains about 33M RGB pixels.

Overall, we explore the joint design of DNN reconstruction algorithms and physics sampling. With our research, we hope to develop more compact, more accurate, and larger covering range computational imaging systems.

Computed tomography (CT) is a medical imaging technique used for the diagnosis and management of numerous conditions, including cancer, fractures, and infections. Automated interpretation of CT scans using deep learning holds immense promise, as it may accelerate the radiology workflow, bring radiology expertise to underserved areas, and reduce missed diagnoses caused by human error. However, several obstacles have thus far prevented deployment of fully automated CT interpretation systems: (1) the difficulty of acquiring and preparing CT volumes; (2) the arduousness of manually acquiring structured abnormality labels needed to train models; (3) the question of how to construct high-performing models for CT interpretation; and (4) the need for explainable models. In this thesis, I address all four challenges.

First, I curated the RAD-ChestCT data set of 36,316 volumes from 19,993 unique patients. I downloaded whole CT volumes in DICOM format using an API developed for the Duke vendor neutral archive. Then I developed the first end-to-end Python pipeline for CT preprocessing, which converts each CT scan from a collection of per-slice DICOM files into a clean 3D NumPy array compatible with major machine learning frameworks. At present, RAD-ChestCT is the largest multiply-annotated volumetric medical imaging data set in the world.

Next, to obtain high-quality labels suitable for training a multiple abnormality classifier, I developed SARLE, a rule-based expert system for automatically extracting abnormality x location labels from free-text radiology reports. SARLE is the first approach to obtain both abnormality and location information from reports, and it obtains high performance, with an average abnormality F-score of 97.6.

A fundamental form of CT interpretation is identification of all abnormalities in the scan. However, prior work has focused on only one class of abnormalities at a time. To address this I developed the CT-Net model, a deep CNN for multiple abnormality prediction from whole volumes. CT-Net achieves an AUROC >90 for 18 abnormalities, with an average AUROC of 77.3 for all 83 abnormalities. Furthermore, training on more abnormalities significantly improves performance. For a subset of 9 labels the model's average AUROC increased by 10% when the number of training labels was increased from 9 to all 83.

One limitation of CT-Net is its lack of explainability. I thus propose AxialNet, a CNN that leverages multiple instance learning to enable identification of key axial slices. Next, I identify a serious problem with the popular model explanation method Grad-CAM: Grad-CAM sometimes creates the false impression that the model has focused on the wrong organ. To address this problem, I propose HiResCAM, a novel attention mechanism for visual explanation in CNNs. I prove that HiResCAM is a generalization of the CAM method and has the intuitive interpretation of highlighting exactly which locations in the input volume lead to an increased score for a particular abnormality, for any CNN ending in a single fully connected layer. Finally, I combine HiResCAM with PARTITION, an approach to obtain allowed regions for each abnormality without manual labeling, to create a mask loss that yields a 37% improvement in organ localization of abnormalities.

Overall, this work advances CNN explanation methods and the clinical applicability of multi-abnormality modeling in volumetric medical images, contributing to the goal of fully automated systems for CT interpretation.