Impressive results in natural language processing (NLP) based on the
Transformer neural network architecture have inspired researchers to explore
viewing offline reinforcement learning (RL) as a generic sequence modeling
problem. Recent works based on this paradigm have achieved state-of-the-art
results in several of the mostly deterministic offline Atari and D4RL
benchmarks. However, because these methods jointly model the states and actions
as a single sequencing problem, they struggle to disentangle the effects of the
policy and world dynamics on the return. Thus, in adversarial or stochastic
environments, these methods lead to overly optimistic behavior that can be
dangerous in safety-critical systems like autonomous driving. In this work, we
propose a method that addresses this optimism bias by explicitly disentangling
the policy and world models, which allows us at test time to search for
policies that are robust to multiple possible futures in the environment. We
demonstrate our method's superior performance on a variety of autonomous
driving tasks in simulation.

本研究提出了一种解决优化偏差的方法，即通过显式分离策略和世界模型，并在测试时寻找能够应对多种可能未来环境的策略，以更好地完成多种自动驾驶任务。