In visual-based Reinforcement Learning (RL), agents often struggle to
generalize well to environmental variations in the state space that were not
observed during training. The variations can arise in both task-irrelevant
features, such as background noise, and task-relevant features, such as robot
configurations, that are related to the optimal decisions. To achieve
generalization in both situations, agents are required to accurately understand
the impact of changed features on the decisions, i.e., establishing the true
associations between changed features and decisions in the policy model.
However, due to the inherent correlations among features in the state space,
the associations between features and decisions become entangled, making it
difficult for the policy to distinguish them. To this end, we propose
Saliency-Guided Features Decorrelation (SGFD) to eliminate these correlations
through sample reweighting. Concretely, SGFD consists of two core techniques:
Random Fourier Functions (RFF) and the saliency map. RFF is utilized to
estimate the complex non-linear correlations in high-dimensional images, while
the saliency map is designed to identify the changed features. Under the
guidance of the saliency map, SGFD employs sample reweighting to minimize the
estimated correlations related to changed features, thereby achieving
decorrelation in visual RL tasks. Our experimental results demonstrate that
SGFD can generalize well on a wide range of test environments and significantly
outperforms state-of-the-art methods in handling both task-irrelevant
variations and task-relevant variations.

我们提出了基于显著性引导的特征解相关方法 (SGFD)，通过样本重新加权消除观测状态空间中特征之间的相关性，以实现在视觉强化学习任务中的泛化能力。实验结果表明，SGFD 在处理任务无关和任务相关的变化方面明显优于现有方法。