Visual Reinforcement Learning (RL) agents trained on limited views face
significant challenges in generalizing their learned abilities to unseen views.
This inherent difficulty is known as the problem of $\textit{view
generalization}$. In this work, we systematically categorize this fundamental
problem into four distinct and highly challenging scenarios that closely
resemble real-world situations. Subsequently, we propose a straightforward yet
effective approach to enable successful adaptation of visual
$\textbf{Mo}$del-based policies for $\textbf{Vie}$w generalization
($\textbf{MoVie}$) during test time, without any need for explicit reward
signals and any modification during training time. Our method demonstrates
substantial advancements across all four scenarios encompassing a total of
$\textbf{18}$ tasks sourced from DMControl, xArm, and Adroit, with a relative
improvement of $\mathbf{33}$%, $\mathbf{86}$%, and $\mathbf{152}$%
respectively. The superior results highlight the immense potential of our
approach for real-world robotics applications. Videos are available at
this https URL .

本文提出了一种名为 MoVie 的视觉模型策略适应方法，通过在测试期间实现视图泛化，无需任何明确的奖励信号和任何训练期间的修改，可显著提高目标任务的性能表现，这表明该方法在实际中应用于机器人技术具有巨大的潜力。

MoVie: 基于模型的视图概括策略自适应

MoVie: Visual Model-Based Policy Adaptation for View Generalization

Humans can easily infer the underlying 3D geometry and texture of an object
only from a single 2D image. Current computer vision methods can do this, too,
but suffer from view generalization problems - the models inferred tend to make
poor predictions of appearance in novel views. As for generalization problems
in machine learning, the difficulty is balancing single-view accuracy (cf.
training error; bias) with novel view accuracy (cf. test error; variance). We
describe a class of models whose geometric rigidity is easily controlled to
manage this tradeoff. We describe a cycle consistency loss that improves view
generalization (roughly, a model from a generated view should predict the
original view well). View generalization of textures requires that models share
texture information, so a car seen from the back still has headlights because
other cars have headlights. We describe a cycle consistency loss that
encourages model textures to be aligned, so as to encourage sharing. We compare
our method against the state-of-the-art method and show both qualitative and
quantitative improvements.

本研究介绍了一种模型，可以在降低模型的偏差的同时提高模型的方差，从而解决计算机视觉中物体单视角的外观推理问题，并加入循环一致性损失来提高视角泛化性能以及纹理对齐，与现有的最先进方法相比，我们的方法在定性和定量方面均有所提高。