Self-supervised representation learning methods mainly focus on image-level
instance discrimination. This study explores the potential benefits of
incorporating patch-level discrimination into existing methods to enhance the
quality of learned representations by simultaneously looking at local and
global visual features. Towards this idea, we present a straightforward yet
effective patch-matching algorithm that can find the corresponding patches
across the augmented views of an image. The augmented views are subsequently
fed into a self-supervised learning framework employing Vision Transformer
(ViT) as its backbone. The result is the generation of both image-level and
patch-level representations. Leveraging the proposed patch-matching algorithm,
the model minimizes the representation distance between not only the CLS tokens
but also the corresponding patches. As a result, the model gains a more
comprehensive understanding of both the entirety of the image as well as its
finer details. We pretrain the proposed method on small, medium, and
large-scale datasets. It is shown that our approach could outperform
state-of-the-art image-level representation learning methods on both image
classification and downstream tasks. Keywords: Self-Supervised Learning; Visual
Representations; Local-Global Representation Learning; Patch-Wise
Representation Learning; Vision Transformer (ViT)

本研究探索了将图像级别与局部级别的判别相结合的自我监督表示学习方法，通过同时观察局部和全局视觉特征来增强学习表示的质量。我们提出了一个简单而有效的补丁匹配算法，借助视觉变换器（ViT）作为骨干网络，从而生成图像级别和补丁级别的表示。结果显示，我们的方法在图像分类和下游任务中表现优于最先进的图像级别表示学习方法。