We develop techniques for refining representations for fine-grained
classification and segmentation tasks in a self-supervised manner. We find that
fine-tuning methods based on instance-discriminative contrastive learning are
not as effective, and posit that recognizing part-specific variations is
crucial for fine-grained categorization. We present an iterative learning
approach that incorporates part-centric equivariance and invariance objectives.
First, pixel representations are clustered to discover parts. We analyze the
representations from convolutional and vision transformer networks that are
best suited for this task. Then, a part-centric learning step aggregates and
contrasts representations of parts within an image. We show that this improves
the performance on image classification and part segmentation tasks across
datasets. For example, under a linear-evaluation scheme, the classification
accuracy of a ResNet50 trained on ImageNet using DetCon, a self-supervised
learning approach, improves from 35.4% to 42.0% on the Caltech-UCSD Birds, from
35.5% to 44.1% on the FGVC Aircraft, and from 29.7% to 37.4% on the Stanford
Cars. We also observe significant gains in few-shot part segmentation tasks
using the proposed technique, while instance-discriminative learning was not as
effective. Smaller, yet consistent, improvements are also observed for stronger
networks based on transformers.

我们基于自监督学习开发了细粒度分类和分割任务的表示优化技术，通过识别部分特异变化改进了 fine-grained 分类，在卷积和视觉变换网络中聚类像素表示，利用部分中心化学习和对比在图像中对部分进行聚类，并在多个数据集上展示了在图像分类和部分分割任务上的性能提升。