Recent advances in visual reinforcement learning (RL) have led to impressive
success in handling complex tasks. However, these methods have demonstrated
limited generalization capability to visual disturbances, which poses a
significant challenge for their real-world application and adaptability. Though
normalization techniques have demonstrated huge success in supervised and
unsupervised learning, their applications in visual RL are still scarce. In
this paper, we explore the potential benefits of integrating normalization into
visual RL methods with respect to generalization performance. We find that,
perhaps surprisingly, incorporating suitable normalization techniques is
sufficient to enhance the generalization capabilities, without any additional
special design. We utilize the combination of two normalization techniques,
CrossNorm and SelfNorm, for generalizable visual RL. Extensive experiments are
conducted on DMControl Generalization Benchmark and CARLA to validate the
effectiveness of our method. We show that our method significantly improves
generalization capability while only marginally affecting sample efficiency. In
particular, when integrated with DrQ-v2, our method enhances the test
performance of DrQ-v2 on CARLA across various scenarios, from 14% of the
training performance to 97%.

本文探讨将规范化技术整合到视觉增强学习方法中，以提高其泛化性能，并使用两种规范化技术，CrossNorm 和 SelfNorm，在 DMControl Generalization Benchmark 和 CARLA 上进行实验证明了该方法的有效性并只会对样本效率略微影响。

规范化增强视觉强化学习的泛化能力

Normalization Enhances Generalization in Visual Reinforcement Learning

Traditional normalization techniques (e.g., Batch Normalization and Instance
Normalization) generally and simplistically assume that training and test data
follow the same distribution. As distribution shifts are inevitable in
real-world applications, well-trained models with previous normalization
methods can perform badly in new environments. Can we develop new normalization
methods to improve generalization robustness under distribution shifts? In this
paper, we answer the question by proposing CrossNorm and SelfNorm. CrossNorm
exchanges channel-wise mean and variance between feature maps to enlarge
training distribution, while SelfNorm uses attention to recalibrate the
statistics to bridge gaps between training and test distributions. CrossNorm
and SelfNorm can complement each other, though exploring different directions
in statistics usage. Extensive experiments on different fields (vision and
language), tasks (classification and segmentation), settings (supervised and
semi-supervised), and distribution shift types (synthetic and natural) show the
effectiveness. Code is available at
this https URL

本文介绍了新的标准化技术 ——CrossNorm 和 SelfNorm，通过交换特征图之间的通道均值和方差以及使用注意力重新校准统计量来改善在 distribution shifts 下的泛化鲁棒性，证明了其在不同领域（视觉和语言），任务（分类和分割），设置（有监督和半监督）和分布转移类型（合成和自然）下非常有效。