We introduce $\mathcal{L}_1$-MBRL, a control-theoretic augmentation scheme for Model-Based Reinforcement Learning (MBRL) algorithms. Unlike model-free approaches, MBRL algorithms learn a model of the transition function using data and use it to design a control input. Our approach generates a series of approximate control-affine models of the learned transition function according to the proposed switching law. Using the approximate model, control input produced by the underlying MBRL is perturbed by the $\mathcal{L}_1$ adaptive control, which is designed to enhance the robustness of the system against uncertainties. Importantly, this approach is agnostic to the choice of MBRL algorithm, enabling the use of the scheme with various MBRL algorithms. MBRL algorithms with $\mathcal{L}_1$ augmentation exhibit enhanced performance and sample efficiency across multiple MuJoCo environments, outperforming the original MBRL algorithms, both with and without system noise.

我们介绍了一种用于模型基强化学习（MBRL）算法的控制论增强方案，该方案通过生成一系列基于学到的转移函数的近似控制仿射模型，并结合L1自适应控制来提高系统的鲁棒性。该方案不依赖于MBRL算法的选择，能够在各种MBRL算法中使用，且在多个MuJoCo环境中表现出卓越的性能和采样效率，优于原始的MBRL算法，无论是否存在系统噪声。

基于鲁棒模型的强化学习在L1自适应控制中的应用