In offline reinforcement learning (RL), the absence of active exploration
calls for attention on the model robustness to tackle the sim-to-real gap,
where the discrepancy between the simulated and deployed environments can
significantly undermine the performance of the learned policy. To endow the
learned policy with robustness in a sample-efficient manner in the presence of
high-dimensional state-action space, this paper considers the sample complexity
of distributionally robust linear Markov decision processes (MDPs) with an
uncertainty set characterized by the total variation distance using offline
data. We develop a pessimistic model-based algorithm and establish its sample
complexity bound under minimal data coverage assumptions, which outperforms
prior art by at least $\tilde{O}(d)$, where $d$ is the feature dimension. We
further improve the performance guarantee of the proposed algorithm by
incorporating a carefully-designed variance estimator.

通过使用离线数据，基于分布健壮的线性马尔科夫决策过程，开发了一种悲观的模型算法，提供了一个具有样本效率的鲁棒性学习策略，以解决离线强化学习中模拟和实际环境之间的差异所带来的问题。