Human beings perceive the world through diverse senses such as sight, smell,
hearing, and touch. Similarly, multimodal large language models (MLLMs) enhance
the capabilities of traditional large language models by integrating and
processing data from multiple modalities including text, vision, audio, video,
and 3D environments. Data plays a pivotal role in the development and
refinement of these models. In this survey, we comprehensively review the
literature on MLLMs from a data-centric perspective. Specifically, we explore
methods for preparing multimodal data during the pretraining and adaptation
phases of MLLMs. Additionally, we analyze the evaluation methods for datasets
and review benchmarks for evaluating MLLMs. Our survey also outlines potential
future research directions. This work aims to provide researchers with a
detailed understanding of the data-driven aspects of MLLMs, fostering further
exploration and innovation in this field.

本综述以数据为中心的视角全面回顾多模态大型语言模型的文献，探索了在多模态数据准备、预训练和适应阶段的方法，分析了数据集的评估方法和评估多模态大型语言模型的基准。此外，本综述还概述了未来的研究方向，以便为研究人员提供对多模态大型语言模型的数据驱动方面的详细理解，推动该领域的进一步探索和创新。

多模态大型语言模型的数据中心视角调查

A Survey of Multimodal Large Language Model from A Data-centric  Perspective

Teacher-Student Curriculum Learning (TSCL) is a curriculum learning framework
that draws inspiration from human cultural transmission and learning. It
involves a teacher algorithm shaping the learning process of a learner
algorithm by exposing it to controlled experiences. Despite its success,
understanding the conditions under which TSCL is effective remains challenging.
In this paper, we propose a data-centric perspective to analyze the underlying
mechanics of the teacher-student interactions in TSCL. We leverage cooperative
game theory to describe how the composition of the set of experiences presented
by the teacher to the learner, as well as their order, influences the
performance of the curriculum that is found by TSCL approaches. To do so, we
demonstrate that for every TSCL problem, there exists an equivalent cooperative
game, and several key components of the TSCL framework can be reinterpreted
using game-theoretic principles. Through experiments covering supervised
learning, reinforcement learning, and classical games, we estimate the
cooperative values of experiences and use value-proportional curriculum
mechanisms to construct curricula, even in cases where TSCL struggles. The
framework and experimental setup we present in this work represent a novel
foundation for a deeper exploration of TSCL, shedding light on its underlying
mechanisms and providing insights into its broader applicability in machine
learning.

基于教师 - 学生课程学习（TSCL）的数据中心视角和合作博弈理论，研究了 TSCL 框架中教师与学生之间的相互作用机制，揭示了经验的组合和顺序对 TSCL 课程的性能产生的影响。通过实验对监督学习、强化学习和经典游戏进行估值和构建课程，为 TSCL 的深入研究提供了新的基础，并揭示了其在机器学习中的广泛适用性。

通过经验的合作机制重新思考教师 - 学生课程学习

Rethinking Teacher-Student Curriculum Learning through the Cooperative  Mechanics of Experience

In real-world applications, machine learning models often become obsolete due
to shifts in the joint distribution arising from underlying temporal trends, a
phenomenon known as the "concept drift". Existing works propose model-specific
strategies to achieve temporal generalization in the near-future domain.
However, the diverse characteristics of real-world datasets necessitate
customized prediction model architectures. To this end, there is an urgent
demand for a model-agnostic temporal domain generalization approach that
maintains generality across diverse data modalities and architectures. In this
work, we aim to address the concept drift problem from a data-centric
perspective to bypass considering the interaction between data and model.
Developing such a framework presents non-trivial challenges: (i) existing
generative models struggle to generate out-of-distribution future data, and
(ii) precisely capturing the temporal trends of joint distribution along
chronological source domains is computationally infeasible. To tackle the
challenges, we propose the COncept Drift simulAtor (CODA) framework
incorporating a predicted feature correlation matrix to simulate future data
for model training. Specifically, CODA leverages feature correlations to
represent data characteristics at specific time points, thereby circumventing
the daunting computational costs. Experimental results demonstrate that using
CODA-generated data as training input effectively achieves temporal domain
generalization across different model architectures.

通过数据为中心的视角解决概念漂移问题，提出了一个模型无关的时间域泛化方法，使用 COncept Drift simulAtor (CODA) 框架通过预测的特征相关矩阵模拟未来数据来训练模型，有效地实现了不同模型结构的时间域泛化。