Personalized federated learning has gained significant attention as a
promising approach to address the challenge of data heterogeneity. In this
paper, we address a relatively unexplored problem in federated learning. When a
federated model has been trained and deployed, and an unlabeled new client
joins, providing a personalized model for the new client becomes a highly
challenging task. To address this challenge, we extend the adaptive risk
minimization technique into the unsupervised personalized federated learning
setting and propose our method, FedTTA. We further improve FedTTA with two
simple yet effective optimization strategies: enhancing the training of the
adaptation model with proxy regularization and early-stopping the adaptation
through entropy. Moreover, we propose a knowledge distillation loss
specifically designed for FedTTA to address the device heterogeneity. Extensive
experiments on five datasets against eleven baselines demonstrate the
effectiveness of our proposed FedTTA and its variants. The code is available
at: this https URL

在个性化联邦学习领域，当一个联邦模型已经训练和部署，并且有一个未标记的新客户端加入时，为新客户端提供个性化模型是一个极具挑战性的任务。本文针对这一挑战，将自适应风险最小化技术扩展到无监督个性化联邦学习设置中，并提出我们的方法 FedTTA。我们通过使用代理正则化和熵提前停止调整来改进 FedTTA，进而提出了两种简单而有效的优化策略。此外，我们还针对 FedTTA 设计了一种专门的知识蒸馏损失，以解决设备异构性问题。针对十一个基准测试，通过对五个数据集进行大量实验，证明了我们提出的 FedTTA 及其变种的有效性。

UPFL：面向新客户的无监督个性化联邦学习

UPFL: Unsupervised Personalized Federated Learning towards New Clients

Lip reading is the task of recognizing the speech content by analyzing
movements in the lip region when people are speaking. Observing on the
continuity in adjacent frames in the speaking process, and the consistency of
the motion patterns among different speakers when they pronounce the same
phoneme, we model the lip movements in the speaking process as a sequence of
apparent deformations in the lip region. Specifically, we introduce a
Deformation Flow Network (DFN) to learn the deformation flow between adjacent
frames, which directly captures the motion information within the lip region.
The learned deformation flow is then combined with the original grayscale
frames with a two-stream network to perform lip reading. Different from
previous two-stream networks, we make the two streams learn from each other in
the learning process by introducing a bidirectional knowledge distillation loss
to train the two branches jointly. Owing to the complementary cues provided by
different branches, the two-stream network shows a substantial improvement over
using either single branch. A thorough experimental evaluation on two
large-scale lip reading benchmarks is presented with detailed analysis. The
results accord with our motivation, and show that our method achieves
state-of-the-art or comparable performance on these two challenging datasets.

本文提出了一种基于变形流网络和双流网络的唇语识别方法，通过引入双向知识蒸馏损失函数让两个分支互相学习，实现了比单一分支更好的性能，并在两个大型唇语识别基准测试中获得了与最先进方法相媲美的结果。