We present a novel prompt-based personalized federated learning (pFL) method
to address data heterogeneity and privacy concerns in traditional medical
visual question answering (VQA) methods. Specifically, we regard medical
datasets from different organs as clients and use pFL to train personalized
transformer-based VQA models for each client. To address the high computational
complexity of client-to-client communication in previous pFL methods, we
propose a succinct information sharing system by introducing prompts that are
small learnable parameters. In addition, the proposed method introduces a
reliability parameter to prevent the negative effects of low performance and
irrelevant clients. Finally, extensive evaluations on various heterogeneous
medical datasets attest to the effectiveness of our proposed method.

我们提出了一种新颖的基于提示的个性化联邦学习（pFL）方法，以解决传统医学视觉问答（VQA）方法中的数据异质性和隐私问题。具体而言，我们将来自不同器官的医学数据集视为客户，并使用 pFL 为每个客户训练个性化的基于 Transformer 的 VQA 模型。为了解决以前的 pFL 方法中客户间通信的高计算复杂性，我们提出了一个简洁的信息共享系统，通过引入小的可学习参数提示。此外，所提出的方法引入了一个可靠性参数，以防止低性能和不相关客户的负面影响。最后，对各种异构医学数据集进行了广泛评估，证明了我们提出的方法的有效性。