In this paper, we present OtterHD-8B, an innovative multimodal model evolved
from Fuyu-8B, specifically engineered to interpret high-resolution visual
inputs with granular precision. Unlike conventional models that are constrained
by fixed-size vision encoders, OtterHD-8B boasts the ability to handle flexible
input dimensions, ensuring its versatility across various inference
requirements. Alongside this model, we introduce MagnifierBench, an evaluation
framework designed to scrutinize models' ability to discern minute details and
spatial relationships of small objects. Our comparative analysis reveals that
while current leading models falter on this benchmark, OtterHD-8B, particularly
when directly processing high-resolution inputs, outperforms its counterparts
by a substantial margin. The findings illuminate the structural variances in
visual information processing among different models and the influence that the
vision encoders' pre-training resolution disparities have on model
effectiveness within such benchmarks. Our study highlights the critical role of
flexibility and high-resolution input capabilities in large multimodal models
and also exemplifies the potential inherent in the Fuyu architecture's
simplicity for handling complex visual data.

OtterHD-8B 是一个可处理高分辨率图像输入的多模态模型，具有灵活的输入维度处理能力，在小物体的细节和空间关系辨析方面的性能超过其他模型。研究结果揭示了不同模型之间在视觉信息处理上的结构差异以及视觉编码器的预训练分辨率差异对模型效果在该基准测试中的影响。该研究突出了大型多模态模型中的灵活性和高分辨率输入能力的关键作用，同时展示了 Fuyu 架构在处理复杂视觉数据方面的潜力。