Incorporating Data Interaction in Deep Learning Architecture, Training, and Applications

Abstract

A central challenge in modern machine learning is to understand and leverage the role of interaction in data—whether between particles, samples, loss surfaces, or signals. Recent theoretical advances have developed principled ways of incorporating interaction directly into the mathematical and statistical foundations of learning. McKean–Vlasov stochastic differential equations (MV-SDEs) capture the dynamics of infinitely many interacting particles by embedding distributional dependence into the system, offering a powerful class of probability flows for modeling exchangeable time series. Building on this foundation, we introduce semi-parametric representations and estimators that explicitly exploit distributional information, yielding improved inference and temporal modeling. Complementing this, we propose PDE-based regularization frameworks that enforce interaction through structural constraints on the loss landscape: parabolic operators that regulate continual learning by bounding forgetting through memory-buffer boundaries, and elliptic operators that enforce smoothness across the data domain to anticipate behavior in regions with limited representation. Together, these theoretical tools highlight how embedding interaction at the level of system dynamics or optimization objectives can improve stability, generalization, and interpretability. We then demonstrate how interaction-centric principles translate into applied domains. Specifically in neuroscience, neural synchrony is shown as a important mechanism to support specific animal behaviors. Inspired by the deep connection between the interacting particle system in MV-SDE and the modern attention mechanism, we model neural synchrony in the insect antennal lobe using an attention-based normalizing flow that captures population-level spike train interactions. The attention weights not only provide semi-interpretable insights into excitation–inhibition balance but also uncover functional roles of specific neuron types in olfactory coding. Across these theoretical and applied settings, a unifying theme emerges: making interactions explicit—whether in probability flows, loss operators, neural activity, or temporal patterns—leads to models that are both more robust and more revealing of underlying structure.