In self-supervised visual representation learning, a feature extractor is
trained on a "pretext task" for which labels can be generated cheaply, without
human annotation. A central challenge in this approach is that the feature
extractor quickly learns to exploit low-level visual features such as color
aberrations or watermarks and then fails to learn useful semantic
representations. Much work has gone into identifying such "shortcut" features
and hand-designing schemes to reduce their effect. Here, we propose a general
framework for mitigating the effect shortcut features. Our key assumption is
that those features which are the first to be exploited for solving the pretext
task may also be the most vulnerable to an adversary trained to make the task
harder. We show that this assumption holds across common pretext tasks and
datasets by training a "lens" network to make small image changes that
maximally reduce performance in the pretext task. Representations learned with
the modified images outperform those learned without in all tested cases.
Additionally, the modifications made by the lens reveal how the choice of
pretext task and dataset affects the features learned by self-supervision.

自我监督视觉表示学习中，通过预训练任务训练特征提取器，提出了使用 lens 网络来减少对抗攻击漏洞，通过对图片进行微小修改，最大限度地减少了自我监督学习中的 “捷径” 特征的影响。