Synthesize 3D realistic CT textures and anatomy in the XCAT phantom using Generative Adversarial Network (GAN)

Abstract

Objectives: Generate 3D realistic textures and anatomy in the thorax region of extended cardiac torso (XCAT) phantom. Methods: We proposed to generate realistic anatomical textures and structures in the XCAT phantom using a conditional generative adversarial network (CGAN) and 3D U-net. 300 and 100 3D CT images were used for training and validation, respectively. Organ maps were generated from patient CT images with uniform intensities in each organ to mimic XCAT images. The model was trained to simulated realistic CT textures and structures in organ maps to match with real CT images. To capture the fine details and guarantee the continuity of lung vessels, we used multiple generators to generate lung vessels and other body parts separately before integrating them together. The results from the training and validation group were evaluated by measuring the SSIM, L1, and L2 metric between the real CT images and simulation images generated from organ maps. After training, XCAT phantoms were input into the model to generate textured XCAT phantoms. The result was compared to our prior work using the 2D GAN model. Results: Compared to the 2D GAN model, the 3D model achieved higher SSIM and PSNR and lower L1 and L2 losses in the validation group. In the generated XCAT phantoms, the model generated similar contrast and anatomical textures as real CT images. The generated phantom also showed 3D continuity by visual examination from different views, especially for lung vessels. Additionally, we found the strategy of using separate generators for lung vessels and other parts greatly enhanced the realism of the generated images.