In the evolution of Vision-Language Pre-training, shifting from short-text
comprehension to encompassing extended textual contexts is pivotal. Recent
autoregressive vision-language models like \cite{flamingo, palme}, leveraging
the long-context capability of Large Language Models, have excelled in few-shot
text generation tasks but face challenges in alignment tasks. Addressing this
gap, we introduce the contrastive loss into text generation models, presenting
the COntrastive-Streamlined MultimOdal framework (\ModelName), strategically
partitioning the language model into dedicated unimodal text processing and
adept multimodal data handling components. \ModelName, our unified framework,
merges unimodal and multimodal elements, enhancing model performance for tasks
involving textual and visual data while notably reducing learnable parameters.
However, these models demand extensive long-text datasets, yet the availability
of high-quality long-text video datasets remains limited. To bridge this gap,
this work introduces \VideoDatasetName, an inaugural interleaved video-text
dataset featuring comprehensive captions, marking a significant step forward.
Demonstrating its impact, we illustrate how \VideoDatasetName{} enhances model
performance in image-text tasks. With 34% learnable parameters and utilizing
72\% of the available data, our model demonstrates significant superiority over
OpenFlamingo~\cite{openflamingo}. For instance, in the 4-shot flickr captioning
task, performance notably improves from 57.2% to 65.\%. The contributions of
\ModelName{} and \VideoDatasetName{} are underscored by notable performance
gains across 14 diverse downstream datasets encompassing both image-text and
video-text tasks.

在视觉语言预训练的演变过程中，从短文理解到包含扩展文本上下文具有关键作用。通过引入对比损失到文本生成模型中，本文提出了一种结合对比学习和多模态处理的统一框架 (ModelName)，在涉及文本和视觉数据的任务中，显著提高模型性能，同时降低学习参数。此外，本文介绍了一个首个交叉视频 - 文本数据集 (VideoDatasetName)，通过全面的字幕，进一步增强模型在图像 - 文本任务中的性能。