In this paper, we focus on learning a linear time-invariant (LTI) model with low-dimensional latent variables but high-dimensional observations. We provide an algorithm that recovers the high-dimensional features, i.e. column space of the observer, embeds the data into low dimensions and learns the low-dimensional model parameters. Our algorithm enjoys a sample complexity guarantee of order $\tilde{\mathcal{O}}(n/\epsilon^2)$, where $n$ is the observation dimension. We further establish a fundamental lower bound indicating this complexity bound is optimal up to logarithmic factors and dimension-independent constants. We show that this inevitable linear factor of $n$ is due to the learning error of the observer's column space in the presence of high-dimensional noise. Extending our results, we consider a meta-learning problem inspired by various real-world applications, where the observer column space can be collectively learned from datasets of multiple LTI systems. An end-to-end algorithm is then proposed, facilitating learning LTI systems from a meta-dataset which breaks the sample complexity lower bound in certain scenarios.

本文研究了一个具有低维潜变量但高维观测值的线性时不变模型，并提出了一个算法，该算法可以恢复高维特征并将数据嵌入到低维空间，并学习低维模型参数，其样本复杂度保证为$\tilde{\mathcal{O}}(n/\epsilon^2)$，其中$n$为观测维度。此外，本文还建立了一个基本的下界，说明该复杂度保证在对数因子和与维度无关的常数上是最优的。本文还研究了一个元学习问题，该问题受到各种实际应用的启发，在这些应用中，观测者的列空间可以从多个线性时不变系统的数据集中共同学习。然后，提出了一种端到端算法，可以从元数据集中学习线性时不变系统，并在某些情况下突破样本复杂度的下界限制。

从高维观测学习低维潜在动态：非渐近性和下界