Building a patient-specific model using transfer learning for 4D-CBCT augmentation

Thumbnail Image




Journal Title

Journal ISSN

Volume Title

Repository Usage Stats



Purpose: Four-dimensional cone beam computed tomography (4D-CBCT) has been developed to provide respiratory phase‐resolved volumetric images in aid of image guided radiation therapy (IGRT), especially in SBRT, which requires highly accurate dose delivery. However, 4D-CBCT suffers from insufficient projection data in each phase bin, which leads to severe noise and artifact. To address this problem, deep learning methods have been introduced to help with augmenting image quality. However, when using traditional deep learning methods to augment CBCT images, the augmented images tend to lose small details such as lung textures. In this study, transfer learning method was proposed to further improve the image quality of the deep-learning augmented CBCT for one specific patient.

Methods: The network architecture used in this project for transfer learning is a standard U-net. CBCT images were reconstructed using limited projections that are simulated from ground truth CT images or directly from clinic. For transfer learning training process, the network was firstly fed with different patients’ data in order to learn a general restoration process to augment under-sampled CBCT images from any patients. Then, the restoration pattern was improved for one specific patient by re-feeding the network with this patient’s data from prior days. Performance of transfer learning was evaluated by comparing the augmented CBCT images to the traditional deep learning method’s images both qualitatively and quantitatively using structure similarity index matrix (SSIM) and peak signal-to-noise ratio (PSNR).

Regarding the study of effectiveness and time efficiency of transfer learning methods, two transfer learning methods, whole-layer fine tuning and layer-freezing methods, are compared to each other. Two training methods, whole-data tuning and sequential tuning were employed as well to further explore the possibility of improving transfer learning’s performance and reducing training time.

Results: The comparison demonstrated that the images augmented from transfer learning method not only recovered more detailed information in lung area but also had more uniform pixel value than basic U-net images when comparing to the ground truth. In addition, two transfer learning methods, whole-layers fine-tuning and layer-freezing method, and two training method, sequential training and all data training, were compared to each other, and all data training with layer-freezing method was found to be time-efficient with training time as short as 10 minutes. In the study of projection number’s effect, transfer-learning augmented CBCT images reconstructed from as low as 90 projection out of 900 projections showed its improvement from U-net augmented images.

Conclusion: Overall, transfer learning based image augmentation method is efficient and effective on improving image qualities of augmented under-sampled 3D/4D-CBCT images from traditional deep-learning methods. Given its relatively fast computational speeds and great performance, it can be very valuable for 4D image guided radiation therapy.





sun, leshan (2020). Building a patient-specific model using transfer learning for 4D-CBCT augmentation. Master's thesis, Duke University. Retrieved from


Dukes student scholarship is made available to the public using a Creative Commons Attribution / Non-commercial / No derivative (CC-BY-NC-ND) license.