The Robust Markov Decision Process (RMDP) framework focuses on designing
control policies that are robust against the parameter uncertainties due to the
mismatches between the simulator model and real-world settings. An RMDP problem
is typically formulated as a max-min problem, where the objective is to find
the policy that maximizes the value function for the worst possible model that
lies in an uncertainty set around a nominal model. The standard robust dynamic
programming approach requires the knowledge of the nominal model for computing
the optimal robust policy. In this work, we propose a model-based reinforcement
learning (RL) algorithm for learning an $\epsilon$-optimal robust policy when
the nominal model is unknown. We consider three different forms of uncertainty
sets, characterized by the total variation distance, chi-square divergence, and
KL divergence. For each of these uncertainty sets, we give a precise
characterization of the sample complexity of our proposed algorithm. In
addition to the sample complexity results, we also present a formal analytical
argument on the benefit of using robust policies. Finally, we demonstrate the
performance of our algorithm on two benchmark problems.

该研究提出了一种基于模型的强化学习算法，用于学习在标准和不确定的模型下最优的稳健控制策略，并考虑了不同形式的不确定性集合