Visual Model-Based Reinforcement Learning (MBRL) promises to encapsulate
agent's knowledge about the underlying dynamics of the environment, enabling
learning a world model as a useful planner. However, top MBRL agents such as
Dreamer often struggle with visual pixel-based inputs in the presence of
exogenous or irrelevant noise in the observation space, due to failure to
capture task-specific features while filtering out irrelevant spatio-temporal
details. To tackle this problem, we apply a spatio-temporal masking strategy, a
bisimulation principle, combined with latent reconstruction, to capture
endogenous task-specific aspects of the environment for world models,
effectively eliminating non-essential information. Joint training of
representations, dynamics, and policy often leads to instabilities. To further
address this issue, we develop a Hybrid Recurrent State-Space Model (HRSSM)
structure, enhancing state representation robustness for effective policy
learning. Our empirical evaluation demonstrates significant performance
improvements over existing methods in a range of visually complex control tasks
such as Maniskill \cite{gu2023maniskill2} with exogenous distractors from the
Matterport environment. Our code is avaliable at
this https URL

利用视觉模型仿真提升强化学习效果，通过空时屏蔽策略和混合循环状态空间模型来准确捕捉环境的重要特征，以提高策略的学习与表现。

学习潜在动态健壮表示的世界模型

Learning Latent Dynamic Robust Representations for World Models

Self-supervised learning has demonstrated remarkable capability in
representation learning for skeleton-based action recognition. Existing methods
mainly focus on applying global data augmentation to generate different views
of the skeleton sequence for contrastive learning. However, due to the rich
action clues in the skeleton sequences, existing methods may only take a global
perspective to learn to discriminate different skeletons without thoroughly
leveraging the local relationship between different skeleton joints and video
frames, which is essential for real-world applications. In this work, we
propose a Partial Spatio-Temporal Learning (PSTL) framework to exploit the
local relationship from a partial skeleton sequences built by a unique
spatio-temporal masking strategy. Specifically, we construct a
negative-sample-free triplet steam structure that is composed of an anchor
stream without any masking, a spatial masking stream with Central Spatial
Masking (CSM), and a temporal masking stream with Motion Attention Temporal
Masking (MATM). The feature cross-correlation matrix is measured between the
anchor stream and the other two masking streams, respectively. (1) Central
Spatial Masking discards selected joints from the feature calculation process,
where the joints with a higher degree of centrality have a higher possibility
of being selected. (2) Motion Attention Temporal Masking leverages the motion
of action and remove frames that move faster with a higher possibility. Our
method achieves SOTA performance on NTU-60, NTU-120 and PKU-MMD under various
downstream tasks. A practical evaluation is performed where some skeleton
joints are lost in downstream tasks. In contrast to previous methods that
suffer from large performance drops, our PSTL can still achieve remarkable
results, validating the robustness of our method. Code:
this https URL

本研究提出了一种利用部分骨骼序列的局部关系的 Partial Spatio-Temporal Learning（PSTL）框架，并采用独特的时空屏蔽策略构建负样本无人机结构。该方法在 NTU-60、NTU-120 和 PKU-MMD 等基准测试中均取得了 SOTA 表现，并在实际评估中验证了方法的鲁棒性。