Large vision-language models are generally applicable to many downstream
tasks, but come at an exorbitant training cost that only large institutions can
afford. This paper trades generality for efficiency and presents Curation in
Training (CiT), a simple and efficient vision-text learning algorithm that
couples a data objective into training. CiT automatically yields quality data
to speed-up contrastive image-text training and alleviates the need for an
offline data filtering pipeline, allowing broad data sources (including raw
image-text pairs from the web). CiT contains two loops: an outer loop curating
the training data and an inner loop consuming the curated training data. The
text encoder connects the two loops. Given metadata for tasks of interest,
e.g., class names, and a large pool of image-text pairs, CiT alternatively
selects relevant training data from the pool by measuring the similarity of
their text embeddings and embeddings of the metadata. In our experiments, we
observe that CiT can speed up training by over an order of magnitude,
especially if the raw data size is large.

本文提出了一种名为 CiT 的有效视觉 - 文本学习算法，通过在训练中将数据目标整合进去，自动产生高质量数据集和提高对比度图像 - 文本训练的速度，从而实现了数据筛选管道的免除和训练速度的提高。