In this report, we introduce MammothModa, yet another multi-modal large
language model (MLLM) designed to achieve state-of-the-art performance starting
from an elementary baseline. We focus on three key design insights: (i)
Integrating Visual Capabilities while Maintaining Complex Language
Understanding: In addition to the vision encoder, we incorporated the Visual
Attention Experts into the LLM to enhance its visual capabilities. (ii)
Extending Context Window for High-Resolution and Long-Duration Visual Feature:
We explore the Visual Merger Module to effectively reduce the token number of
high-resolution images and incorporated frame position ids to avoid position
interpolation. (iii) High-Quality Bilingual Datasets: We meticulously curated
and filtered a high-quality bilingual multimodal dataset to reduce visual
hallucinations. With above recipe we build MammothModa that consistently
outperforms the state-of-the-art models, e.g., LLaVA-series, across main
real-world visual language benchmarks without bells and whistles.

我们介绍了 MammothModa，这是另一个多模态大型语言模型（MLLM），旨在从基本基线实现最先进的性能。我们关注了三个关键设计见解：（i）在保持复杂语言理解的同时融入视觉能力：除了视觉编码器外，我们还将视觉注意力专家纳入 LLM 以增强其视觉能力。（ii）扩展上下文窗口以获得高分辨率和长持续时间的视觉特征：我们探索了视觉合并模块，以有效地减少高分辨率图像的标记数量，并引入了帧位置 ID 以避免位置插值。（iii）高质量的双语数据集：我们精心策划和筛选了一个高质量的双模态双语数据集，以减少视觉幻觉。通过上述方法，我们构建的 MammothModa 在主要的真实世界视觉语言基准测试中始终优于最先进的模型，例如 LLaVA 系列模型，而没有花里胡哨的东西。