We consider a collaborative learning setting where each agent's goal is to
improve their own model by leveraging the expertise of collaborators, in
addition to their own training data. To facilitate the exchange of expertise
among agents, we propose a distillation-based method leveraging unlabeled
auxiliary data, which is pseudo-labeled by the collective. Central to our
method is a trust weighting scheme which serves to adaptively weigh the
influence of each collaborator on the pseudo-labels until a consensus on how to
label the auxiliary data is reached. We demonstrate that our collaboration
scheme is able to significantly boost individual model's performance with
respect to the global distribution, compared to local training. At the same
time, the adaptive trust weights can effectively identify and mitigate the
negative impact of bad models on the collective. We find that our method is
particularly effective in the presence of heterogeneity among individual
agents, both in terms of training data as well as model architectures.

本文提出了基于蒸馏的方法，并采用信任加权方案，使用伪标签的无标签辅助数据促进合作学习，以达成对辅助数据的标签共识。我们证明了该方案相对于本地训练能显著提高模型性能，同时也可以识别和缓解对于众多模型的负面影响。在个体数据训练以及模型架构存在异质性的情况下，该方法表现得尤为有效。