Artificial Intelligence for Intelligent Computer Architecture

Abstract

Machine learning and artificial intelligence models have profoundly transformed human society, impacting everything from daily life to the automation of industrial production. Recent advancements in large language models have further expanded the applications of machine learning, enhancing fields such as artistic creation and scientific research.Throughout the history of machine learning and artificial intelligence, scaling has proven to be the most successful approach to advancing model capabilities. This has driven the continuous growth of model sizes, with the largest models now consisting of hundreds of billions of parameters. The increasing parameter size has created unprecedented computational challenges, leading to an urgent need for innovative computer architectures to meet the demands of large machine learning and AI models.In this thesis, we explore a novel approach that leverages artificial intelligence models themselves to design intelligent computer architectures for AI. To meet the scaling demands of models, we claim that not only should we design architectures that enable the scaling to larger models, but the scalability of the hardware design methodologies themselves is also crucial. We start by concentrating on the architectural design itself, exploring architectural enhancements for efficient deployment of large models. This includes dataflow optimizations, specialized numeric datatypes, and compression-enabled architectures to enhance efficiency. Then, we reverse our approach by revisiting the design methodology of computer architectures. In this context, we demonstrate how to leverage machine learning models to guide, assist, and accelerate the hardware architecture development flow. Finally, we showcase the benefits of our proposed methods and how they can be applied to real-world hardware development processes.