Leveraging Metaplasticity and Physical Constraints for Reliable AI

Abstract

Deep learning methods have achieved considerable success in cognitive tasks that require large amounts of data due to their extreme flexibility, provided by millions of trainable parameters. However, this flexibility comes at the expense of making deep learning models prone to overfitting their training data. As a result, models become vulnerable to noise-based adversarial attacks, develop rigid decision boundaries, and struggle to tolerate unexpected inputs. Accounting for internal noise and dynamics in the data is crucial for creating reliable deep learning systems, especially those designed to perform critical tasks in which data may be scarce or continuously updated. This thesis develops tools for creating reliable deep learning methods across the algorithmic, hardware-software co-design, and device domains.We design machine learning methods that leverage uncertainty in training data to improve robustness against corrupted samples, gracefully adapt to unseen conditions, and generalize the underlying dynamics of physical systems. In real-world applications, machine learning models will inevitably encounter noisy or novel data and must remain reliable under such conditions. This motivates the study of metaplastic learning rules and physical constraints. We introduce one perspective on metaplastic learning, a second on physics-informed learning, and a third that integrates both. First, we consider metaplastic learning rules from an algorithmic perspective through Bayesian inference. Second, we examine metaplastic learning rules through a hardware-software co-design grounded in ferroelectric field-effect transistors. Lastly, we contextualize physics-informed learning to model the behaviors of ultrathin HfO2-ZrO2 films for nonvolatile capacitors.In our first thrust, we introduce a method for accelerating variational Bayesian learning for image classifiers by leveraging synaptic uncertainty. We extend a metaplastic learning rule to operate on any deep neural network through backpropagated errors and uncertainty estimates. This formulation reframes Bayesian neural network training as maximizing the test accuracy while minimizing the variability of the learned parameter distributions. By allowing uncertainty to tune individual synaptic learning rates, the model naturally performs early stopping as confidence increases and demonstrates superior robustness against gradient-based adversarial attacks.Our second thrust presents a hardware-software co-design that implements probabilistic metaplasticity. We exploit the stochastic switching properties of ferroelectric field-effect transistors to realize probabilistic bits that drive weight consolidation for continual learning. Embedding stochastic updates driven by probabilistic bits into the crossbar architecture enables scalable parallel in-memory learning while reducing circuit overhead and improving energy savings compared to existing methods. By using this consolidation mechanism to freeze important weights, the system preserves previously acquired knowledge and remains robust to the non-idealities inherent in nanoscale ferroelectric domains.In our third thrust, we provide a case study in physics-informed learning from a compact modeling perspective. Ultrathin HfO2-ZrO2 films exhibit ferroelectric and antiferroelectric properties, enabling the fabrication of transistors and capacitors that are promising candidates for emulating synaptic and neuronal dynamics in neuromorphic circuits. However, their history-dependent hysteresis and rate-sensitive switching make these devices difficult to model, and the complexity of traditional models scales with the richness of the behaviors they attempt to reproduce, leading to computationally cumbersome simulations. We show that training long short-term memory models with the Landau-Khalatnikov equation as a physics-based loss term allows the model to capture essential device behaviors while achieving unprecedented generalization on unseen input patterns.We conclude by charting a path forward for Bayesian inference, continual learning, and compact modeling. Our work establishes a foundation for efficient hardware-software codesign in probabilistic machine learning and for incorporating richer dynamics into physics informed long short-term memory models. By addressing reliability across three levels of abstraction, we pave the way for models and systems that remain robust to noisy inputs and constrain themselves to prevent producing unexpected outputs.