The fusion of complementary multimodal information is crucial in
computational pathology for accurate diagnostics. However, existing multimodal
learning approaches necessitate access to users' raw data, posing substantial
privacy risks. While Federated Learning (FL) serves as a privacy-preserving
alternative, it falls short in addressing the challenges posed by heterogeneous
(yet possibly overlapped) modalities data across various hospitals. To bridge
this gap, we propose a Federated Multi-Modal (FedMM) learning framework that
federatedly trains multiple single-modal feature extractors to enhance
subsequent classification performance instead of existing FL that aims to train
a unified multimodal fusion model. Any participating hospital, even with
small-scale datasets or limited devices, can leverage these federated trained
extractors to perform local downstream tasks (e.g., classification) while
ensuring data privacy. Through comprehensive evaluations of two publicly
available datasets, we demonstrate that FedMM notably outperforms two baselines
in accuracy and AUC metrics.

提出并评估了一种名为 FedMM 的联邦多模态学习框架，该框架通过联邦训练多个单模态特征提取器来增强后续分类性能，以解决计算病理学中多模态信息融合的隐私问题。