Effectively leveraging multimodal data such as various images, laboratory
tests and clinical information is gaining traction in a variety of AI-based
medical diagnosis and prognosis tasks. Most existing multi-modal techniques
only focus on enhancing their performance by leveraging the differences or
shared features from various modalities and fusing feature across different
modalities. These approaches are generally not optimal for clinical settings,
which pose the additional challenges of limited training data, as well as being
rife with redundant data or noisy modality channels, leading to subpar
performance. To address this gap, we study the robustness of existing methods
to data redundancy and noise and propose a generalized dynamic multimodal
information bottleneck framework for attaining a robust fused feature
representation. Specifically, our information bottleneck module serves to
filter out the task-irrelevant information and noises in the fused feature, and
we further introduce a sufficiency loss to prevent dropping of task-relevant
information, thus explicitly preserving the sufficiency of prediction
information in the distilled feature. We validate our model on an in-house and
a public COVID19 dataset for mortality prediction as well as two public
biomedical datasets for diagnostic tasks. Extensive experiments show that our
method surpasses the state-of-the-art and is significantly more robust, being
the only method to remain performance when large-scale noisy channels exist.
Our code is publicly available at this https URL

利用多模态数据，在医学诊断和预测任务中有效利用各种图像、实验室测试和临床信息的人工智能技术正在发展。本研究通过研究现有方法对数据冗余和噪声的鲁棒性，提出了一种广义的动态多模态信息瓶颈框架，用于获得稳健的融合特征表示。通过大量实验证明，我们的方法超越了现有技术，在存在大规模噪声通道时仍然具有优越性能。