Transformers have achieved remarkable success in various machine-learning
tasks, prompting their widespread adoption. In this paper, we explore their
application in the context of federated learning (FL), with a particular focus
on heterogeneous scenarios where individual clients possess diverse local
datasets. To meet the computational and communication demands of FL, we
leverage pre-trained Transformers and use an efficient prompt-tuning strategy.
Our strategy introduces the concept of learning both shared and group prompts,
enabling the acquisition of universal knowledge and group-specific knowledge
simultaneously. Additionally, a prompt selection module assigns personalized
group prompts to each input, aligning the global model with the data
distribution of each client. This approach allows us to train a single global
model that can automatically adapt to various local client data distributions
without requiring local fine-tuning. In this way, our proposed method
effectively bridges the gap between global and personalized local models in
Federated Learning and surpasses alternative approaches that lack the
capability to adapt to previously unseen clients. The effectiveness of our
approach is rigorously validated through extensive experimentation and ablation
studies.

在联邦学习中，我们利用预训练的 Transformer 和高效的提示调整策略，通过引入学习共享和组特定提示的概念，使全局模型能够自动适应各种局部客户数据分布，从而有效地弥合全局和个性化本地模型之间的差距，并超越以往无法适应之前未见客户的其他方法。

具有群组感知提示调整的异构联邦学习

Heterogeneous Federated Learning with Group-Aware Prompt Tuning

Zero-shot cross-domain slot filling aims to transfer knowledge from the
labeled source domain to the unlabeled target domain. Existing models either
encode slot descriptions and examples or design handcrafted question templates
using heuristic rules, suffering from poor generalization capability or
robustness. In this paper, we propose a generative zero-shot prompt learning
framework for cross-domain slot filling, both improving generalization and
robustness than previous work. Besides, we introduce a novel inverse prompting
strategy to distinguish different slot types to avoid the multiple prediction
problem, and an efficient prompt-tuning strategy to boost higher performance by
only training fewer prompt parameters. Experiments and analysis demonstrate the
effectiveness of our proposed framework, especially huge improvements (+13.44%
F1) on the unseen slots.

零 - shot 跨领域槽填充的生成式零 - shot 提示学习框架，通过反向提示策略区分不同槽类型以避免多重预测问题，以及通过训练更少的提示参数来提高性能，证明了我们提出的框架的有效性，尤其是在未见过的槽上有巨大的提升（+13.44% F1）。