Efficiency in Security-preserving Training of Artificial Intelligence

Abstract

The rapid development of AI systems has also exposed significant security vulnerabilities, particularly in robustness and privacy. This dissertation focuses on security-preserving training, a class of training-stage defenses for the security of AI systems. While extensive research has justified the defensive effectiveness of security-preserving training methods, these approaches often suffer from inefficiency, characterized by slower convergence compared to standard training algorithms. Despite some progress in analyzing specific security-preserving training algorithms, a comprehensive theoretical framework to explain and address this inefficiency remains absent.This dissertation addresses this gap by quantitatively analyzing the efficiency of security-preserving training algorithms from an optimization perspective. I derive a theorem on the convergence of generalized gradient descent, revealing that training efficiency is closely tied to the concept of gradient shifting, i.e., the deviation between the descent direction and the gradient direction. Gradient shifting is further decomposed into distribution inconsistency, arising from training data manipulation, and objective inconsistency, resulting from modifications to the training loss. This theoretical framework can be applied to various security-preserving training algorithms, providing insights into improving efficiency without compromising security. Additionally, the theory can also be applied reversely to develop defense mechanisms that hinder the training of adversarial attackers, thereby enhancing AI system security.Three case studies are presented in this dissertation with applications of the theory in different scenarios: (1) FedCor accelerates the convergence of federated learning by mitigating the objective inconsistency with an active client selection; (2) FedProphet tackles the objective inconsistency caused by systematic heterogeneity and perturbed data in federated adversarial training; (3) ModelGuard hinders the model extraction attack by enlarging the distribution inconsistency in knowledge distillation. The effective application of the theory in these case studies validates its versatility across various contexts.