Large web crawl datasets have already played an important role in learning
multimodal features with high generalization capabilities. However, there are
still very limited studies investigating the details or improvements of data
design. Recently, a DataComp challenge has been designed to propose the best
training data with the fixed models. This paper presents our solution to both
filtering track and BYOD track of the DataComp challenge. Our solution adopts
large multimodal models CLIP and BLIP-2 to filter and modify web crawl data,
and utilize external datasets along with a bag of tricks to improve the data
quality. Experiments show our solution significantly outperforms DataComp
baselines (filtering track: 6.6% improvement, BYOD track: 48.5% improvement).

本文介绍了我们在 DataComp 挑战赛的筛选和修改轨道上的解决方案，采用大型多模态模型 CLIP 和 BLIP-2 对网络爬行数据进行过滤和修改，并利用外部数据集和一系列技巧来提高数据质量。实验证明我们的解决方案在 DataComp 基线的基础上取得了显著的进展（筛选轨道：改善了 6.6％，BYOD 轨道：改善了 48.5％）。