The generalization of representations learned via contrastive learning
depends crucially on what features of the data are extracted. However, we
observe that the contrastive loss does not always sufficiently guide which
features are extracted, a behavior that can negatively impact the performance
on downstream tasks via "shortcuts", i.e., by inadvertently suppressing
important predictive features. We find that feature extraction is influenced by
the difficulty of the so-called instance discrimination task (i.e., the task of
discriminating pairs of similar points from pairs of dissimilar ones). Although
harder pairs improve the representation of some features, the improvement comes
at the cost of suppressing previously well represented features. In response,
we propose implicit feature modification (IFM), a method for altering positive
and negative samples in order to guide contrastive models towards capturing a
wider variety of predictive features. Empirically, we observe that IFM reduces
feature suppression, and as a result improves performance on vision and medical
imaging tasks. The code is available at: https://github.com/joshr17/IFM.

提出了一种名为隐式特征修改 (IFM) 的方法，以改进对比学习的特征提取，避免压抑重要的预测特征并提高视觉和医学成像任务的性能。