Collaborative personalization, such as through learned user representations
(embeddings), can improve the prediction accuracy of neural-network-based
models significantly. We propose Federated User Representation Learning (FURL),
a simple, scalable, privacy-preserving and resource-efficient way to utilize
existing neural personalization techniques in the Federated Learning (FL)
setting. FURL divides model parameters into federated and private parameters.
Private parameters, such as private user embeddings, are trained locally, but
unlike federated parameters, they are not transferred to or averaged on the
server. We show theoretically that this parameter split does not affect
training for most model personalization approaches. Storing user embeddings
locally not only preserves user privacy, but also improves memory locality of
personalization compared to on-server training. We evaluate FURL on two
datasets, demonstrating a significant improvement in model quality with 8% and
51% performance increases, and approximately the same level of performance as
centralized training with only 0% and 4% reductions. Furthermore, we show that
user embeddings learned in FL and the centralized setting have a very similar
structure, indicating that FURL can learn collaboratively through the shared
parameters while preserving user privacy.

本文介绍了一种名为 FURL 的方法，它使用现有的神经个性化技术，通过学习用户表示（嵌入）来改善神经网络模型预测准确性。FURL 将模型参数分为联邦和私有参数，其中私有参数（如私有用户嵌入）在本地训练，但不会被传输或平均在服务器上。该参数拆分对大多数模型个性化方法的训练没有影响。在两个数据集上评估 FURL，表明模型质量有了显着提高，并且用户嵌入在 FL 和集中式设置中具有非常相似的结构，表明 FURL 可以通过共享参数进行协作学习，同时保护用户隐私。