While image data starts to enjoy the simple-but-effective self-supervised
learning scheme built upon masking and self-reconstruction objective thanks to
the introduction of tokenization procedure and vision transformer backbone,
convolutional neural networks as another important and widely-adopted
architecture for image data, though having contrastive-learning techniques to
drive the self-supervised learning, still face the difficulty of leveraging
such straightforward and general masking operation to benefit their learning
process significantly. In this work, we aim to alleviate the burden of
including masking operation into the contrastive-learning framework for
convolutional neural networks as an extra augmentation method. In addition to
the additive but unwanted edges (between masked and unmasked regions) as well
as other adverse effects caused by the masking operations for ConvNets, which
have been discussed by prior works, we particularly identify the potential
problem where for one view in a contrastive sample-pair the randomly-sampled
masking regions could be overly concentrated on important/salient objects thus
resulting in misleading contrastiveness to the other view. To this end, we
propose to explicitly take the saliency constraint into consideration in which
the masked regions are more evenly distributed among the foreground and
background for realizing the masking-based augmentation. Moreover, we introduce
hard negative samples by masking larger regions of salient patches in an input
image. Extensive experiments conducted on various datasets, contrastive
learning mechanisms, and downstream tasks well verify the efficacy as well as
the superior performance of our proposed method with respect to several
state-of-the-art baselines.

通过在卷积神经网络中引入掩蔽操作作为额外的数据增强方法，并显式考虑显著性约束，本研究提出了一种改进的对比学习框架，以减轻包括掩蔽操作在内的对比学习中存在的问题，并在多个数据集、对比学习机制和下游任务上进行的广泛实验验证了其有效性和卓越性能。