Multimodal interleaved datasets featuring free-form interleaved sequences of
images and text are crucial for training frontier large multimodal models
(LMMs). Despite the rapid progression of open-source LMMs, there remains a
pronounced scarcity of large-scale, diverse open-source multimodal interleaved
datasets. In response, we introduce MINT-1T, the most extensive and diverse
open-source Multimodal INTerleaved dataset to date. MINT-1T comprises one
trillion text tokens and three billion images, a 10x scale-up from existing
open-source datasets. Additionally, we include previously untapped sources such
as PDFs and ArXiv papers. As scaling multimodal interleaved datasets requires
substantial engineering effort, sharing the data curation process and releasing
the dataset greatly benefits the community. Our experiments show that LMMs
trained on MINT-1T rival the performance of models trained on the previous
leading dataset, OBELICS. Our data and code will be released at
this https URL

介绍了迄今为止最广泛且多样化的开源多模态交错数据集 MINT-1T，其中包括十亿个文本标记和三十亿个图像，并分享了在该数据集上训练的大型多模态模型 LMMs 的性能与之前领先数据集 OBELICS 相媲美。