Multimodal models are expected to be a critical component to future advances
in artificial intelligence. This field is starting to grow rapidly with a surge
of new design elements motivated by the success of foundation models in natural
language processing (NLP) and vision. It is widely hoped that further extending
the foundation models to multiple modalities (e.g., text, image, video, sensor,
time series, graph, etc.) will ultimately lead to generalist multimodal models,
i.e. one model across different data modalities and tasks. However, there is
little research that systematically analyzes recent multimodal models
(particularly the ones that work beyond text and vision) with respect to the
underling architecture proposed. Therefore, this work provides a fresh
perspective on generalist multimodal models (GMMs) via a novel architecture and
training configuration specific taxonomy. This includes factors such as
Unifiability, Modularity, and Adaptability that are pertinent and essential to
the wide adoption and application of GMMs. The review further highlights key
challenges and prospects for the field and guide the researchers into the new
advancements.

多模态模型在人工智能的未来进展中被认为是一个关键组成部分，本文通过一种新的架构和训练配置特定的分类方法，从多个角度对通用多模态模型进行了综述，并提出了关键挑战和前景。