In the current landscape of foundation model training, there is a significant
reliance on public domain data, which is nearing exhaustion according to recent
research. To further scale up, it is crucial to incorporate collaboration among
multiple specialized and high-quality private domain data sources. However, the
challenge of training models locally without sharing private data presents
numerous obstacles in data quality control. To tackle this issue, we propose a
data quality control pipeline for federated fine-tuning of foundation models.
This pipeline computes scores reflecting the quality of training data and
determines a global threshold for a unified standard, aiming for improved
global performance. Our experiments show that the proposed quality control
pipeline facilitates the effectiveness and reliability of the model training,
leading to better performance.

当前基础模型训练的情况表明，公共领域数据已接近枯竭，因此需要在多个专门化和高质量的私有领域数据源之间加强合作。为了解决本地训练模型而不共享私有数据所带来的质量控制问题，我们提出了一个针对基础模型的联邦微调的数据质量控制流程。该流程通过计算反映训练数据质量的分数，并确定一个统一标准的全局阈值，旨在提高整体性能。我们的实验证明，所提出的质量控制流程有助于模型训练的效果和可靠性，从而提高性能。