We present a new publicly available dataset with the goal of advancing
multi-modality learning by offering vision and language data within the same
context. This is achieved by obtaining data from a social media website with
posts containing multiple paired images/videos and text, along with comment
trees containing images/videos and/or text. With a total of 677k posts, 2.9
million post images, 488k post videos, 1.4 million comment images, 4.6 million
comment videos, and 96.9 million comments, data from different modalities can
be jointly used to improve performances for a variety of tasks such as image
captioning, image classification, next frame prediction, sentiment analysis,
and language modeling. We present a wide range of statistics for our dataset.
Finally, we provide baseline performance analysis for one of the regression
tasks using pre-trained models and several fully connected networks.

该论文介绍了一个通过从社交媒体网站获取包含多个成对图像 / 视频和文本的帖子以及包含图像 / 视频和 / 或文本的评论树来实现多模态学习的新的可公开获取的数据集，并提供了基线性能分析，适用于图像字幕、情感分析和预训练模型等多项任务。