We introduce InternLM-XComposer2, a cutting-edge vision-language model
excelling in free-form text-image composition and comprehension. This model
goes beyond conventional vision-language understanding, adeptly crafting
interleaved text-image content from diverse inputs like outlines, detailed
textual specifications, and reference images, enabling highly customizable
content creation. InternLM-XComposer2 proposes a Partial LoRA (PLoRA) approach
that applies additional LoRA parameters exclusively to image tokens to preserve
the integrity of pre-trained language knowledge, striking a balance between
precise vision understanding and text composition with literary talent.
Experimental results demonstrate the superiority of InternLM-XComposer2 based
on InternLM2-7B in producing high-quality long-text multi-modal content and its
exceptional vision-language understanding performance across various
benchmarks, where it not only significantly outperforms existing multimodal
models but also matches or even surpasses GPT-4V and Gemini Pro in certain
assessments. This highlights its remarkable proficiency in the realm of
multimodal understanding. The InternLM-XComposer2 model series with 7B
parameters are publicly available at
this https URL

InternLM-XComposer2 是一种先进的视觉 - 语言模型，擅长自由形式的文本 - 图像组合和理解，在从轮廓、详细的文本规范和参考图像等多样输入中巧妙地创建交错的文本 - 图像内容，实现了高度可定制的内容创作。InternLM-XComposer2 提出了 “Partial LoRA (PLoRA)” 方法，将额外的 LoRA 参数专门应用于图像标记，以保留预训练语言知识的完整性，在精确的视觉理解和具有文学才能的文本组合之间取得平衡。实验结果表明，在产生高质量的长文本多模态内容和在各种基准测试中的出色视觉 - 语言理解性能方面，InternLM-XComposer2 基于 InternLM2-7B 模型具有明显优势，不仅显著优于现有的多模态模型，而且在某些评估中甚至超过 GPT-4V 和 Gemini Pro，凸显了它在多模态理解领域的卓越能力。InternLM-XComposer2 系列模型的参数为 7B，可公开获取。