Self-supervised learning is a powerful paradigm for representation learning
on unlabelled images. A wealth of effective new methods based on instance
matching rely on data-augmentation to drive learning, and these have reached a
rough agreement on an augmentation scheme that optimises popular recognition
benchmarks. However, there is strong reason to suspect that different tasks in
computer vision require features to encode different (in)variances, and
therefore likely require different augmentation strategies. In this paper, we
measure the invariances learned by contrastive methods and confirm that they do
learn invariance to the augmentations used and further show that this
invariance largely transfers to related real-world changes in pose and
lighting. We show that learned invariances strongly affect downstream task
performance and confirm that different downstream tasks benefit from polar
opposite (in)variances, leading to performance loss when the standard
augmentation strategy is used. Finally, we demonstrate that a simple fusion of
representations with complementary invariances ensures wide transferability to
all the diverse downstream tasks considered.

本文研究了自监督学习在图像中表征学习的应用，通过对比实例匹配方法，我们发现不同的视觉任务需要不同的数据扩充策略，并且使用具有互补不变性的表征方法可以提高各种下游任务的表现。

为什么自监督模型能够迁移？—— 探究不变性在下游任务中的影响

Why Do Self-Supervised Models Transfer? Investigating the Impact of  Invariance on Downstream Tasks

Sensing surroundings plays a crucial role in human spatial perception, as it
extracts the spatial configuration of objects as well as the free space from
the observations. To facilitate the robot perception with such a surrounding
sensing capability, we introduce a novel visual task called Cross-view Semantic
Segmentation as well as a framework named View Parsing Network (VPN) to address
it. In the cross-view semantic segmentation task, the agent is trained to parse
the first-view observations into a top-down-view semantic map indicating the
spatial location of all the objects at pixel-level. The main issue of this task
is that we lack the real-world annotations of top-down-view data. To mitigate
this, we train the VPN in 3D graphics environment and utilize the domain
adaptation technique to transfer it to handle real-world data. We evaluate our
VPN on both synthetic and real-world agents. The experimental results show that
our model can effectively make use of the information from different views and
multi-modalities to understanding spatial information. Our further experiment
on a LoCoBot robot shows that our model enables the surrounding sensing
capability from 2D image input. Code and demo videos can be found at
https://view-parsing-network.github.io.

为了提高机器人感知周围环境的能力，本研究提出了一种名为 Cross-view Semantic Segmentation 的视觉任务并推出了 View Parsing Network（VPN）框架来解决该任务，该任务能够将第一视角的观测数据转化为顶视图的物体语义分割图，表示出所有物体在像素层级的空间位置。通过在 3D 图形环境中训练 VPN 并使用域自适应技术将其转移到处理现实环境的数据，本研究证明了在实验中，VPN 显著提高了机器人理解空间信息的能力。