Despite noise and caption quality having been acknowledged as important
factors impacting vision-language contrastive pre-training, in this paper, we
show that the full potential of improving the training process by addressing
such issues is yet to be realized. Specifically, we firstly study and analyze
two issues affecting training: incorrect assignment of negative pairs, and low
caption quality and diversity. Then, we devise effective solutions for
addressing both problems, which essentially require training with multiple true
positive pairs. Finally, we propose training with sigmoid loss to address such
a requirement. We show very large gains over the current state-of-the-art for
both image recognition ($\sim +6\%$ on average over 11 datasets) and image
retrieval ($\sim +19\%$ on Flickr30k and $\sim +15\%$ on MSCOCO).

本文研究视觉语言对比预训练中的问题，提出了解决负样本分配不正确和字幕质量低和多样性不足的有效方法，并通过使用 sigmoid loss 进行训练，在图像识别和图像检索方面取得了非常大的增益。