We introduce Xmodel-VLM, a cutting-edge multimodal vision language model. It
is designed for efficient deployment on consumer GPU servers. Our work directly
confronts a pivotal industry issue by grappling with the prohibitive service
costs that hinder the broad adoption of large-scale multimodal systems. Through
rigorous training, we have developed a 1B-scale language model from the ground
up, employing the LLaVA paradigm for modal alignment. The result, which we call
Xmodel-VLM, is a lightweight yet powerful multimodal vision language model.
Extensive testing across numerous classic multimodal benchmarks has revealed
that despite its smaller size and faster execution, Xmodel-VLM delivers
performance comparable to that of larger models. Our model checkpoints and code
are publicly available on GitHub at this https URL

我们介绍了 Xmodel-VLM，这是一款先进的多模态视觉语言模型，旨在在消费级 GPU 服务器上进行高效部署。我们的工作直接应对了一个关键行业问题，即解决了巨大规模多模态系统普及所面临的高昂服务成本限制。通过严格的训练，我们从头开始开发了一个 10 亿级的语言模型，采用了 LLaVA 模式用于模态对齐。结果是，我们称之为 Xmodel-VLM 的模型，它既轻量又强大。通过在多个经典多模态基准测试中进行了全面的测试，我们发现尽管模型体积较小且执行速度更快，但 Xmodel-VLM 的性能与较大模型相当。我们的模型检查点和代码公开在 GitHub 上提供。

Xmodel-VLM: 一个简单的多模态视觉语言模型基准

Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model

We present MobileVLM, a competent multimodal vision language model (MMVLM)
targeted to run on mobile devices. It is an amalgamation of a myriad of
architectural designs and techniques that are mobile-oriented, which comprises
a set of language models at the scale of 1.4B and 2.7B parameters, trained from
scratch, a multimodal vision model that is pre-trained in the CLIP fashion,
cross-modality interaction via an efficient projector. We evaluate MobileVLM on
several typical VLM benchmarks. Our models demonstrate on par performance
compared with a few much larger models. More importantly, we measure the
inference speed on both a Qualcomm Snapdragon 888 CPU and an NVIDIA Jeston Orin
GPU, and we obtain state-of-the-art performance of 21.5 tokens and 65.3 tokens
per second, respectively. Our code will be made available at:
this https URL

我们提出了 MobileVLM，这是一个专为移动设备设计的多模式视觉语言模型（MMVLM）。它集成了各种移动设备定向的架构设计和技术，包括一组从头训练的 1.4B 和 2.7B 参数规模的语言模型，以及使用 CLIP 风格预训练的多模式视觉模型，通过高效的投影实现跨模态交互。我们在几个典型的 VLM 基准测试上评估了 MobileVLM。与一些更大的模型相比，我们的模型表现出与之相当的性能。更重要的是，我们在高通骁龙 888 CPU 和 NVIDIA Jeston Orin GPU 上测量了推断速度，分别获得了 21.5 个 token 和 65.3 个 token 每秒的最新性能。我们的代码将在此 https URL 上提供。