From Spectral Theorem to Spectral Statistics of Large Random Matrices with Spatio-Temporal Dependencies

Abstract

High dimensional random dynamical systems are ubiquitous, including-but not limited to- cyber-physical systems, daily return on different stocks of S\&P 1500 and velocity profile of interacting particle systems around McKeanVlasov limit. Mathematically speaking, observed time series data can be captured via a stable $n-$ dimensional linear transformation `$A$' and additive randomness. System identification aims at extracting useful information about underlying dynamical system, given a length $N$ trajectory from it (corresponds to an $n \times N$ dimensional data matrix). We use spectral theorem for non-Hermitian operators to show that spatio-temperal correlations are dictated by the \emph{discrepancy between algebraic andgeometric multiplicity of distinct eigenvalues} corresponding to state transition matrix. Small discrepancies imply that original trajectory essentially comprises of multiple \emph{lower dimensional random dynamical systems living on $A$ invariant subspaces and are statistically independent of each other}. In the process, we provide first quantitative handle on decay rate of finite powers of state transition matrix $\|A^{k}\|$ . It is shown that when a stable dynamical system has only one distinct eigenvalue and discrepancy of $n-1$: $\|A\|$ has a dependence on $n$, resulting dynamics are \emph{spatially inseparable} and consequently there exist at least one row with covariates of typical size $\Theta\big(\sqrt{N-n+1}$ $e^{n}\big)$ i.e., even under stability assumption, covariates can \emph{suffer from curse of dimensionality }. In the light of these findings we set the stage for non-asymptotic error analysis in estimation of state transition matrix $A$ via least squares regression on observed trajectory by showing that element-wise error is essentially a variant of well-know Littlewood-Offord problem and(can be extremely sensitive to dimension of the state space and number of iterations). We also show that largest singular value of the data matrix can be cursed by dimensionality even when state-transition matrix is stable. Overarching theme of this thesis is new theoretical results on spectral theorem for non-Hermitian operators, non-asymptotic behavior of high dimensional dynamical systems , which we incorporate with the work of Talagrand on concentration of measure phenomenon to better understand behavior of the structured random matrices(data matrix) and subsequently the performance of different learning algorithms with dependent data. Besides, we also show that there exists stable linear Gaussians with process level Talagrands' inequality linear in dimension of the state space(previously an open problem), along with deterioration of mixing times with increase in discrepancy between algebraic and geometric multiplicity of $A$.