We consider the Linear-Quadratic-Regulator (LQR) problem in terms of
optimizing a real-valued matrix function over the set of feedback gains. Such a
setup facilitates examining the implications of a natural initial-state
independent formulation of LQR in designing first order algorithms. It is shown
that this cost function is smooth and coercive, and provide an alternate means
of noting its gradient dominated property. In the process, we provide a number
of analytic observations on the LQR cost when directly analyzed in terms of the
feedback gain. We then examine three types of well-posed flows for LQR:
gradient flow, natural gradient flow and the quasi-Newton flow. The coercive
property suggests that these flows admit unique solutions while gradient
dominated property indicates that the corresponding Lyapunov functionals decay
at an exponential rate; we also prove that these flows are exponentially stable
in the sense of Lyapunov. We then discuss the forward Euler discretization of
these flows, realized as gradient descent, natural gradient descent and the
quasi-Newton iteration. We present stepsize criteria for gradient descent and
natural gradient descent, guaranteeing that both algorithms converge linearly
to the global optima. An optimal stepsize for the quasi-Newton iteration is
also proposed, guaranteeing a $Q$-quadratic convergence rate--and in the
meantime--recovering the Hewer algorithm.

该论文探讨了在反馈增益范围内最大化一个实值矩阵函数的线性二次调节器问题，提出了几种算法，包括基于梯度的方法和牛顿迭代算法。研究表明，这些算法均具有一定的收敛性和稳定性。

一阶方法视角下的 LQR 控制：离散时间情形

LQR through the Lens of First Order Methods: Discrete-time Case

High performance tracking control can only be achieved if a good model of the
dynamics is available. However, such a model is often difficult to obtain from
first order physics only. In this paper, we develop a data-driven control law
that ensures closed loop stability of Lagrangian systems. For this purpose, we
use Gaussian Process regression for the feed-forward compensation of the
unknown dynamics of the system. The gains of the feedback part are adapted
based on the uncertainty of the learned model. Thus, the feedback gains are
kept low as long as the learned model describes the true system sufficiently
precisely. We show how to select a suitable gain adaption law that incorporates
the uncertainty of the model to guarantee a globally bounded tracking error. A
simulation with a robot manipulator demonstrates the efficacy of the proposed
control law.

使用基于数据的控制法律，通过高斯过程回归来补偿系统未知的动力学，并基于学习的模型的不确定性来自适应反馈部分的增益，以保证全局有界跟踪误差。