Multimodal machine learning has gained significant attention in recent years
due to its potential for integrating information from multiple modalities to
enhance learning and decision-making processes. However, it is commonly
observed that unimodal models outperform multimodal models, despite the latter
having access to richer information. Additionally, the influence of a single
modality often dominates the decision-making process, resulting in suboptimal
performance. This research project aims to address these challenges by
proposing a novel regularization term that encourages multimodal models to
effectively utilize information from all modalities when making decisions. The
focus of this project lies in the video-audio domain, although the proposed
regularization technique holds promise for broader applications in embodied AI
research, where multiple modalities are involved. By leveraging this
regularization term, the proposed approach aims to mitigate the issue of
unimodal dominance and improve the performance of multimodal machine learning
systems. Through extensive experimentation and evaluation, the effectiveness
and generalizability of the proposed technique will be assessed. The findings
of this research project have the potential to significantly contribute to the
advancement of multimodal machine learning and facilitate its application in
various domains, including multimedia analysis, human-computer interaction, and
embodied AI research.

通过提出新的正则化项，促进多模态模型在决策过程中有效利用所有模态的信息，以缓解单模态主导性问题并提高多模态机器学习系统的性能。