Browsing by Subject "Artificial Intelligence"

Deep learning has demonstrated remarkable success in developing vision intelligence across a variety of application domains, including autonomous driving, facial recognition, medical image analysis, \etc.However, developing such vision systems poses significant challenges, particularly in relation to ensuring efficiency, interpretability, and fairness. Efficiency requires a model to leverage the least possible computational resources while preserving performance relative to more computationally-demanding alternatives, which is essential for the practical deployment of large-scale models in real-time applications. Interpretability demands a model to align with the domain-specific knowledge of the task it addresses while having the capability for case-based reasoning. This characteristic is especially crucial in high-stakes areas such as healthcare, criminal justice, and financial investment. Fairness ensures that computer vision models do not perpetuate or exacerbate societal biases in downstream applications such as web image search, text-guided image generation, \etc. In this dissertation, I will discuss the contributions that I have made in advancing vision intelligence regarding to efficiency, interpretability and fairness in computer vision models.

The first part of this dissertation will focus on how to design computer vision models to efficiently process very large images.We propose a novel CNN architecture termed { \em Zoom-In Network} that leverages a hierarchical attention sampling mechanisms to select important regions of images to process. Such approach without processing the entire image yields outstanding memory efficiency while maintaining classification accuracy on various tiny object image classification datasets.

The second part of this dissertation will discuss how to build post-hoc interpretation method for deep learning models to obtain insights reasoned from the predictions.We propose a novel image and text insight-generation framework based on attributions from deep neural nets. We test our approach on an industrial dataset and demonstrate our method outperforms competing methods.

Finally, we study fairness in large vision-language models.More specifically, we examined gender and racial bias in text-based image retrieval for neutral text queries. In an attempt to address bias in the test-time phase, we proposed post-hoc bias mitigation to actively balance the demographic group in the image search results. Experiments on multiple datasets show that our method can significantly reduce bias while maintaining satisfactory retrieval accuracy at the same time.

My research in enhancing vision intelligence via developments in efficiency, interpretability, and fairness, has undergone rigorous validation using publicly available benchmarks and has been recognized at leading peer-reviewed machine learning conferences.This dissertation has sparked interest within the AI community, emphasizing the importance of improving computer vision models through these three critical dimensions, namely, efficiency, interpretability and fairness.

Objectives

Methods

Results

Conclusions

Purpose

Methods

Results

Conclusions

Level of evidence iv

Objective

Methods

Results

Conclusions

Purpose

Methods

Results

Conclusions

Clinical relevance

Background

Results

Conclusions

Introduction: The current head-and-neck (HN) fluence map generator tends to producehighly modulated fluence maps and therefore high monitor units (MUs) for each beam, which leads to more delivery uncertainty and leakage dose. This project implements diffu- sion into the training process and modifies the loss functions to mitigate this effect.

Methods: The dataset consists of 200 head-and-neck (HN) patients receiving intensity mod-ulated radiation therapy (IMRT) for training, 16 for validation, and 15 for testing. Two models were trained, one with-diffusion and one without. The original model was a con- ditional generative adversarial network (GAN) written in TensorFlow, the model without diffusion was written to be the PyTorch equivalent of the original model. After confirming the model was properly converted to PyTorch by comparing outputs, both new models were modified to use binary cross entropy for the GAN loss and mean absolute error as a third loss function for the generator. Hyperparameters were carefully selected based on the training script for the original model, and further tuned with trial and error. The diffusion was implemented based on Diffusion-GAN and the associated GitHub repository. The two new models were compared by plotting training loss vs epoch over 500 epochs. The two models were compared to the original model by comparing the output fluence maps to the ground truth using similarity index and comparing DVH statistics among the three models.

Results: The with-diffusion model and no-diffusion model achieved similar training loss.The diffusion model and no-diffusion model consistently delivered better parotid sparing than the original model and delivered less dose to four of the six tested OAR. The with- diffusion model delivered less dose to five of the six tested OAR. The diffusion model had the least MUs: 23% less than the original model and 3% less than the no-diffusion model. The diffusion model had lower D2cc: 4% less than the original model and 1% less than the no-diffusion model on average. All three plans deliver 95% of the prescription dose to nearly the same percentage of PTV volume.

Conclusion: Implementing diffusion does not provide a significant impact on training timeand training loss. However, it does enable comparable dose performance to both the no- diffusion and original models, while significantly reducing the total MU’s and 3D max 2cc relative to the original model and slightly reducing these metrics relative to the no-diffusion model, indicating smoother fluence modulation. In addition, both new models reduced dose to the right and left parotids relative to the original model, and to four of six tested OAR total, while the with-diffusion model consistently delivers less dose to OAR than the no- diffusion model. This indicates that both the new loss functions and diffusion reduce the overall dose to the OARs while preserving dose conformity around the target.