Federated Domain-specific Instruction Tuning (FedDIT) utilizes limited cross-client private data together with server-side public data for instruction augmentation, ultimately boosting model performance within specific domains. To date, the factors affecting FedDIT remain unclear, and existing instruction augmentation methods primarily focus on the centralized setting without considering distributed environments. Our experiments reveal that the cross-client domain coverage, rather than data heterogeneity, drives model performance in FedDIT. In response, we propose FedDCA, which optimizes domain coverage through greedy client center selection and retrieval-based augmentation. For client-side computational efficiency and system scalability, FedDCA$^*$, the variant of FedDCA, utilizes heterogeneous encoders with server-side feature alignment. Extensive experiments across four distinct domains (code, medical, financial, and mathematical) substantiate the effectiveness of both methods. Additionally, we investigate privacy preservation against memory extraction attacks utilizing various amounts of public data. Results show that there is no significant correlation between the volume of public data and the privacy-preserving capability. However, as the fine-tuning rounds increase, the risk of privacy leakage reduces or converges.

本研究解决了联邦领域特定指令调优（FedDIT）中影响模型性能的因素尚不明确的问题，提出了利用跨客户端领域覆盖优化的新方法FedDCA，这通过贪婪选择客户端中心和基于检索的增强来实现。实验结果表明，该方法在多个领域（代码、医学、金融和数学）的有效性显著提升，同时探讨了在不同公共数据量下的隐私保护能力的影响。

联邦指令调优中的领域覆盖增强