The exploration of multimodal language models integrates multiple data types,
such as images, text, language, audio, and other heterogeneity. While the
latest large language models excel in text-based tasks, they often struggle to
understand and process other data types. Multimodal models address this
limitation by combining various modalities, enabling a more comprehensive
understanding of diverse data. This paper begins by defining the concept of
multimodal and examining the historical development of multimodal algorithms.
Furthermore, we introduce a range of multimodal products, focusing on the
efforts of major technology companies. A practical guide is provided, offering
insights into the technical aspects of multimodal models. Moreover, we present
a compilation of the latest algorithms and commonly used datasets, providing
researchers with valuable resources for experimentation and evaluation. Lastly,
we explore the applications of multimodal models and discuss the challenges
associated with their development. By addressing these aspects, this paper aims
to facilitate a deeper understanding of multimodal models and their potential
in various domains.

本文探究了多模态语言模型的研究，集成了多种数据类型，如图像、文本、语言、音频和其他异构数据。通过合并各种模态，多模态模型能够更全面地理解和处理多样化的数据，本文旨在促进对多模态模型及其在各领域中的潜力的更深入理解。