Vision Transformers (ViT), when paired with large-scale pretraining, have
shown remarkable performance across various computer vision tasks, primarily
due to their weak inductive bias. However, while such weak inductive bias aids
in pretraining scalability, this may hinder the effective adaptation of ViTs
for visuo-motor control tasks as a result of the absence of control-centric
inductive biases. Such absent inductive biases include spatial locality and
translation equivariance bias which convolutions naturally offer. To this end,
we introduce Convolution Injector (CoIn), an add-on module that injects
convolutions which are rich in locality and equivariance biases into a
pretrained ViT for effective adaptation in visuo-motor control. We evaluate
CoIn with three distinct types of pretrained ViTs (CLIP, MVP, VC-1) across 12
varied control tasks within three separate domains (Adroit, MetaWorld, DMC),
and demonstrate that CoIn consistently enhances control task performance across
all experimented environments and models, validating the effectiveness of
providing pretrained ViTs with control-centric biases.

使用 Convolution Injector（CoIn）给训练有素的 Vision Transformers（ViTs）注入富含局部性和等变性的卷积，提高其在视觉运动控制方面的适应性和性能。

使用卷积注入器适应预训练 ViTs 的视觉动作控制

Adapting Pretrained ViTs with Convolution Injector for Visuo-Motor  Control

End-to-end reinforcement learning on images showed significant progress in
the recent years. Data-based approach leverage data augmentation and domain
randomization while representation learning methods use auxiliary losses to
learn task-relevant features. Yet, reinforcement still struggles in visually
diverse environments full of distractions and spurious noise. In this work, we
tackle the problem of robust visual control at its core and present VIBR
(View-Invariant Bellman Residuals), a method that combines multi-view training
and invariant prediction to reduce out-of-distribution (OOD) generalization gap
for RL based visuomotor control. Our model-free approach improve baselines
performances without the need of additional representation learning objectives
and with limited additional computational cost. We show that VIBR outperforms
existing methods on complex visuo-motor control environment with high visual
perturbation. Our approach achieves state-of the-art results on the Distracting
Control Suite benchmark, a challenging benchmark still not solved by current
methods, where we evaluate the robustness to a number of visual perturbators,
as well as OOD generalization and extrapolation capabilities.

本文提出基于多视角训练和不变预测的 VIBR 算法，从根本上解决具有视觉干扰的复杂环境下强化学习及视觉运动控制的鲁棒性问题，并在 Distracting Control Suite 基准测试中取得了最优结果。

VIBR：学习视角不变值函数进行鲁棒视觉控制

VIBR: Learning View-Invariant Value Functions for Robust Visual Control

Poor sample efficiency continues to be the primary challenge for deployment
of deep Reinforcement Learning (RL) algorithms for real-world applications, and
in particular for visuo-motor control. Model-based RL has the potential to be
highly sample efficient by concurrently learning a world model and using
synthetic rollouts for planning and policy improvement. However, in practice,
sample-efficient learning with model-based RL is bottlenecked by the
exploration challenge. In this work, we find that leveraging just a handful of
demonstrations can dramatically improve the sample-efficiency of model-based
RL. Simply appending demonstrations to the interaction dataset, however, does
not suffice. We identify key ingredients for leveraging demonstrations in model
learning -- policy pretraining, targeted exploration, and oversampling of
demonstration data -- which forms the three phases of our model-based RL
framework. We empirically study three complex visuo-motor control domains and
find that our method is 150%-250% more successful in completing sparse reward
tasks compared to prior approaches in the low data regime (100K interaction
steps, 5 demonstrations). Code and videos are available at:
this https URL

利用演示可以显著提高模型学习效率，在这项工作中，我们确定了利用演示进行模型学习的关键因素，即策略预训练，有针对性的探索和演示数据的过采样，这三个阶段构成了我们的基于模型的 RL 框架。