Wasserstein distributionally robust estimators have emerged as powerful
models for prediction and decision-making under uncertainty. These estimators
provide attractive generalization guarantees: the robust objective obtained
from the training distribution is an exact upper bound on the true risk with
high probability. However, existing guarantees either suffer from the curse of
dimensionality, are restricted to specific settings, or lead to spurious error
terms. In this paper, we show that these generalization guarantees actually
hold on general classes of models, do not suffer from the curse of
dimensionality, and can even cover distribution shifts at testing. We also
prove that these results carry over to the newly-introduced regularized
versions of Wasserstein distributionally robust problems.

本研究表明，Wasserstein 分布鲁棒估计器的推广保证其可适用于一般模型类，并且不会受到维度诅咒，甚至可以涵盖测试中的分布偏移，这些结果可以延伸到新引入的 Wasserstein 分布性机器学习问题的正则化版本。

（正则化的）Wasserstein 鲁棒分布模型精确泛化保证

Exact Generalization Guarantees for (Regularized) Wasserstein  Distributionally Robust Models

Distributionally Robust Optimization (DRO) has enabled to prove the
equivalence between robustness and regularization in classification and
regression, thus providing an analytical reason why regularization generalizes
well in statistical learning. Although DRO's extension to sequential
decision-making overcomes $\textit{external uncertainty}$ through the robust
Markov Decision Process (MDP) setting, the resulting formulation is hard to
solve, especially on large domains. On the other hand, existing regularization
methods in reinforcement learning only address $\textit{internal uncertainty}$
due to stochasticity. Our study aims to facilitate robust reinforcement
learning by establishing a dual relation between robust MDPs and
regularization. We introduce Wasserstein distributionally robust MDPs and prove
that they hold out-of-sample performance guarantees. Then, we introduce a new
regularizer for empirical value functions and show that it lower bounds the
Wasserstein distributionally robust value function. We extend the result to
linear value function approximation for large state spaces. Our approach
provides an alternative formulation of robustness with guaranteed finite-sample
performance. Moreover, it suggests using regularization as a practical tool for
dealing with $\textit{external uncertainty}$ in reinforcement learning methods.

本文介绍了一种新的 Wasserstein 分布强化学习框架，并将其与正则化相联系，从而为解决强化学习中的外部不确定性问题提供一个新的实用工具。

强化学习中的分布鲁棒性和正则化

Distributional Robustness and Regularization in Reinforcement Learning

Standard stochastic control methods assume that the probability distribution
of uncertain variables is available. Unfortunately, in practice, obtaining
accurate distribution information is a challenging task. To resolve this issue,
we investigate the problem of designing a control policy that is robust against
errors in the empirical distribution obtained from data. This problem can be
formulated as a two-player zero-sum dynamic game problem, where the action
space of the adversarial player is a Wasserstein ball centered at the empirical
distribution. We propose computationally tractable value and policy iteration
algorithms with explicit estimates of the number of iterations required for
constructing an $\epsilon$-optimal policy. We show that the contraction
property of associated Bellman operators extends a single-stage out-of-sample
performance guarantee, obtained using a measure concentration inequality, to
the corresponding multi-stage guarantee without any degradation in the
confidence level. In addition, we characterize an explicit form of the optimal
distributionally robust control policy and the worst-case distribution policy
for linear-quadratic problems with Wasserstein penalty. Our study indicates
that dynamic programming and Kantorovich duality play a critical role in
solving and analyzing the Wasserstein distributionally robust stochastic
control problems.

研究了一个基于 Wasserstein 分布的鲁棒控制策略问题，提出了一个可计算的值迭代算法和策略迭代算法，并通过动态规划和 Kantorovich 对偶理论的分析，在保证置信水平不降低的情况下，构造了一个多阶段性能保证和最优分布鲁棒控制策略。