We present the TinyLLaVA framework that provides a unified perspective in
designing and analyzing the small-scale Large Multimodal Models (LMMs). We
empirically study the effects of different vision encoders, connection modules,
language models, training data and training recipes. Our extensive experiments
showed that better quality of data combined with better training recipes,
smaller LMMs can consistently achieve on-par performances compared to bigger
LMMs. Under our framework, we train a family of small-scale LMMs. Our best
model, TinyLLaVA-3.1B, achieves better overall performance against existing 7B
models such as LLaVA-1.5 and Qwen-VL. We hope our findings can serve as
baselines for future research in terms of data scaling, training setups and
model selections. Our model weights and codes will be made public.

通过 TinyLLaVA 框架的研究与实验，我们发现数据的质量、训练配方和模型选择对于设计和分析小规模的大型多模型模型非常重要；通过将更好的数据质量和更好的训练配方相结合，我们发现较小规模的模型可以与更大规模的模型达到相当的性能，这些发现可为未来的研究提供基准。