Federated learning is a training paradigm that learns from multiple
distributed users without aggregating data on a centralized server. Such a
paradigm promises the ability to deploy machine-learning at-scale to a diverse
population of end-users without first collecting a large, labeled dataset for
all possible tasks. As federated learning typically averages learning updates
across a decentralized population, there is a growing need for personalization
of federated learning systems (i.e conversational agents must be able to
personalize to a specific user's preferences). In this work, we propose a new
direction for personalization research within federated learning, leveraging
both personal embeddings and shared context embeddings. We also present an
approach to predict these ``preference'' embeddings, enabling personalization
without backpropagation. Compared to state-of-the-art personalization
baselines, our approach achieves a 50\% improvement in test-time perplexity
using 0.001\% of the memory required by baseline approaches, and achieving
greater sample- and compute-efficiency.

本文提出了利用个人 embedding 和共享上下文 embedding 的方法来进行联邦学习中的个性化研究，并提出了一种预测 “偏好” embedding 的方法，相对于现有方法在测试时间困惑度方面有 50% 的改进，且消耗的内存和计算量更小。