Model predictive control can optimally deal with nonlinear systems under
consideration of constraints. The control performance depends on the model
accuracy and the prediction horizon. Recent advances propose to use
reinforcement learning applied to a parameterized model predictive controller
to recover the optimal control performance even if an imperfect model or short
prediction horizons are used. However, common reinforcement learning algorithms
rely on first order updates, which only have a linear convergence rate and
hence need an excessive amount of dynamic data. Higher order updates are
typically intractable if the policy is approximated with neural networks due to
the large number of parameters.
In this work, we use a parameterized model predictive controller as policy,
and leverage the small amount of necessary parameters to propose a trust-region
constrained Quasi-Newton training algorithm for policy optimization with a
superlinear convergence rate. We show that the required second order derivative
information can be calculated by the solution of a linear system of equations.
A simulation study illustrates that the proposed training algorithm outperforms
other algorithms in terms of data efficiency and accuracy.

通过使用参数化模型预测控制器作为策略并利用所需参数的少量，我们提出了一种带有超线性收敛率的限制拟牛顿训练算法进行策略优化。通过解线性方程组的解来计算所需的二阶导数信息。模拟研究表明，所提出的训练算法在数据效率和准确性方面优于其他算法。