Multimodal learning, which integrates data from diverse sensory modes, plays
a pivotal role in artificial intelligence. However, existing multimodal
learning methods often struggle with challenges where some modalities appear
more dominant than others during multimodal learning, resulting in suboptimal
performance. To address this challenge, we propose MLA (Multimodal Learning
with Alternating Unimodal Adaptation). MLA reframes the conventional joint
multimodal learning process by transforming it into an alternating unimodal
learning process, thereby minimizing interference between modalities.
Simultaneously, it captures cross-modal interactions through a shared head,
which undergoes continuous optimization across different modalities. This
optimization process is controlled by a gradient modification mechanism to
prevent the shared head from losing previously acquired information. During the
inference phase, MLA utilizes a test-time uncertainty-based model fusion
mechanism to integrate multimodal information. Extensive experiments are
conducted on five diverse datasets, encompassing scenarios with complete
modalities and scenarios with missing modalities. These experiments demonstrate
the superiority of MLA over competing prior approaches.

MLA 是通过交替的单模态适应过程改变传统联合多模态学习过程，减少模态之间的干扰，并通过共享头部实现跨模态交互，从而优化性能，在推理阶段利用基于不确定性的模型融合机制集成多模态信息。经过在五个不同数据集上的广泛实验，证明了 MLA 相对于竞争方法的优越性。