We propose methods to strengthen the invariance properties of representations
obtained by contrastive learning. While existing approaches implicitly induce a
degree of invariance as representations are learned, we look to more directly
enforce invariance in the encoding process. To this end, we first introduce a
training objective for contrastive learning that uses a novel regularizer to
control how the representation changes under transformation. We show that
representations trained with this objective perform better on downstream tasks
and are more robust to the introduction of nuisance transformations at test
time. Second, we propose a change to how test time representations are
generated by introducing a feature averaging approach that combines encodings
from multiple transformations of the original input, finding that this leads to
across the board performance gains. Finally, we introduce the novel Spirograph
dataset to explore our ideas in the context of a differentiable generative
process with multiple downstream tasks, showing that our techniques for
learning invariance are highly beneficial.

本文主要介绍了强化对比学习得到的表示的不变性的方法，通过引入新的正则化器、特征平均法和可微分生成过程，对旋转等变性进行限制和控制，提高了表示在后续任务中的表现和鲁棒性。