Visual recognition is recently learned via either supervised learning on
human-annotated image-label data or language-image contrastive learning with
webly-crawled image-text pairs. While supervised learning may result in a more
discriminative representation, language-image pretraining shows unprecedented
zero-shot recognition capability, largely due to the different properties of
data sources and learning objectives. In this work, we introduce a new
formulation by combining the two data sources into a common image-text-label
space. In this space, we propose a new learning paradigm, called Unified
Contrastive Learning (UniCL) with a single learning objective to seamlessly
prompt the synergy of two data types. Extensive experiments show that our UniCL
is an effective way of learning semantically rich yet discriminative
representations, universally for image recognition in zero-shot, linear-probe,
fully finetuning and transfer learning scenarios. Particularly, it attains
gains up to 9.2% and 14.5% in average on zero-shot recognition benchmarks over
the language-image contrastive learning and supervised learning methods,
respectively. In linear probe setting, it also boosts the performance over the
two methods by 7.3% and 3.4%, respectively. Our study also indicates that UniCL
stand-alone is a good learner on pure image-label data, rivaling the supervised
learning methods across three image classification datasets and two types of
vision backbones, ResNet and Swin Transformer. Code is available at
this https URL.

该研究提出了一种新型的学习范式 - 统一对比学习（UniCL），通过将人类标注的图像标签数据和网络爬取的图像文本数据相结合，学习出在零样本，线性探测，完全微调和迁移学习方案中具有语义丰富而有区分性的表示。在各种基准测试中，UniCL 的性能均优于语言图像对比学习和监督学习方法，并且在纯图像标签数据上，其表现也不亚于监督学习方法。