We abstract the features~(\textit{i.e.} learned representations) of
multi-modal data into 1)~\emph{uni-modal features}, which can be learned from
uni-modal training, and 2) \emph{paired features}, which can \emph{only} be
learned from cross-modal interactions. Multi-modal models are expected to
benefit from cross-modal interactions on the basis of ensuring uni-modal
feature learning. However, recent supervised multi-modal late-fusion training
approaches still suffer from insufficient learning of uni-modal features on
each modality. \emph{We prove that this phenomenon does hurt the model's
generalization ability}. To this end, we propose to choose a targeted
late-fusion learning method for the given supervised multi-modal task from
\textbf{U}ni-\textbf{M}odal \textbf{E}nsemble~(UME) and the proposed
\textbf{U}ni-\textbf{M}odal \textbf{T}eacher~(UMT), according to the
distribution of uni-modal and paired features. We demonstrate that, under a
simple guiding strategy, we can achieve comparable results to other complex
late-fusion or intermediate-fusion methods on various multi-modal datasets,
including VGG-Sound, Kinetics-400, UCF101, and ModelNet40.

通过提出一种针对多模态学习的目标后融合方法并使用 Uni-Modal Ensemble (UME) 和 Uni-Modal Teacher (UMT) 的策略来得出 Uni-Modal 特征和 paired 特征的分布，可以在各种多模态数据集上实现与其他复杂的后融合或中间融合方法可比的结果。我们证明，缺乏在每个模态上进行 Uni-modal feature learning 的现象将确实损害模型的泛化能力。