Improving PET Image Using U-net and MRI

Abstract

This study aims to develop a modified deep learning convolutional network algorithm, a U-net, to improve the quality of a PET image. In this study, the U-net was designed to accept a PET image and a corresponding MRI images as inputs, and make a prediction of improved PET image based on the inputs. The PET images are created by tracers that are bound in different concentrations to different tissue types, such as [18F]FDG or a tracer designed to detect amyloid plaques. The MRI image should be taken either at the same time or at a recent (prior or post) time when the PET image is taken. A U-net that was originally designed to predict black and white binary segmentation maps was constructed. It was then modified to output grey-scale non binary images with pixels of different color intensities. Digital phantoms were used to input image datasets for U-net. To improve upon the limitation of previous studies in this area, this study used relatively realistic digital phantoms to generate datasets (as opposed to using clinical data, physical phantoms, or simplistic digital phantoms). The modified U-nets were then trained with generated training datasets using a PC with Intel Core i7-9700K CPU, NVIDIA GTX2080Ti GPU, and 16GB memory. Each resulting trained U-net was used to process 1000 random datasets, and the results were evaluated using DSC or SSIM index based on the output image type of U-nets to see if the predictions are improved compared with the input The trained U-net successfully produced improved images of PET tracer distribution, and showed that it is possible for a U-net to locate irregular lesions of varying textures. This result also showed that a U-net trained with simulated data can produce practical results (predictions with statistically significant improvement) when operating on images resulted from a realistic digital phantom.