In this work, we study model-based reinforcement learning (RL) in unknown
stabilizable linear dynamical systems. When learning a dynamical system, one
needs to stabilize the unknown dynamics in order to avoid system blow-ups. We
propose an algorithm that certifies fast stabilization of the underlying system
by effectively exploring the environment with an improved exploration strategy.
We show that the proposed algorithm attains $\tilde{\mathcal{O}}(\sqrt{T})$
regret after $T$ time steps of agent-environment interaction. We also show that
the regret of the proposed algorithm has only a polynomial dependence in the
problem dimensions, which gives an exponential improvement over the prior
methods. Our improved exploration method is simple, yet efficient, and it
combines a sophisticated exploration policy in RL with an isotropic exploration
strategy to achieve fast stabilization and improved regret. We empirically
demonstrate that the proposed algorithm outperforms other popular methods in
several adaptive control tasks.

研究模型基于的强化学习在未知可稳定线性动态系统中的应用，提出一种通过改进探索策略证明基本稳定性的算法，所提出的算法在避免系统崩溃的同时，实现了对环境的快速探索，在多个自适应控制任务中表现优异。

线性动态系统中带快速稳定的强化学习

Reinforcement Learning with Fast Stabilization in Linear Dynamical  Systems

While deep feature learning has revolutionized techniques for static-image
understanding, the same does not quite hold for video processing. Architectures
and optimization techniques used for video are largely based off those for
static images, potentially underutilizing rich video information. In this work,
we rethink both the underlying network architecture and the stochastic learning
paradigm for temporal data. To do so, we draw inspiration from classic theory
on linear dynamic systems for modeling time series. By extending such models to
include nonlinear mappings, we derive a series of novel recurrent neural
networks that sequentially make top-down predictions about the future and then
correct those predictions with bottom-up observations. Predictive-corrective
networks have a number of desirable properties: (1) they can adaptively focus
computation on "surprising" frames where predictions require large corrections,
(2) they simplify learning in that only "residual-like" corrective terms need
to be learned over time and (3) they naturally decorrelate an input data stream
in a hierarchical fashion, producing a more reliable signal for learning at
each layer of a network. We provide an extensive analysis of our lightweight
and interpretable framework, and demonstrate that our model is competitive with
the two-stream network on three challenging datasets without the need for
computationally expensive optical flow.

本研究提出了一种基于线性动态系统和非线性映射理论的预测 - 纠正神经网络模型，能够自适应地关注‘惊奇’的视频帧，优化学习效率。研究表明，该模型在三个挑战数据集上的表现与双流网络相当，而无需计算昂贵的光流技术。