Federated learning (FL) is a machine learning paradigm in which distributed
local nodes collaboratively train a central model without sharing individually
held private data. Existing FL methods either iteratively share local model
parameters or deploy co-distillation. However, the former is highly susceptible
to private data leakage, and the latter design relies on the prerequisites of
task-relevant real data. Instead, we propose a data-free FL framework based on
local-to-central collaborative distillation with direct input and output space
exploitation. Our design eliminates any requirement of recursive local
parameter exchange or auxiliary task-relevant data to transfer knowledge,
thereby giving direct privacy control to local users. In particular, to cope
with the inherent data heterogeneity across locals, our technique learns to
distill input on which each local model produces consensual yet unique results
to represent each expertise. Our proposed FL framework achieves notable
privacy-utility trade-offs with extensive experiments on image classification
and segmentation tasks under various real-world heterogeneous federated
learning settings on both natural and medical images.

基于本地到中心的协作蒸馏的无数据联邦学习框架，通过学习对每个本地模型产生一致但独特结果的输入，实现对各种实际世界异构联邦学习场景中的图像分类和分割任务的显著隐私 - 效用权衡。

通过输入输出协作蒸馏的联邦学习

Federated Learning via Input-Output Collaborative Distillation

Although the facial makeup transfer network has achieved high-quality
performance in generating perceptually pleasing makeup images, its capability
is still restricted by the massive computation and storage of the network
architecture. We address this issue by compressing facial makeup transfer
networks with collaborative distillation and kernel decomposition. The main
idea of collaborative distillation is underpinned by a finding that the
encoder-decoder pairs construct an exclusive collaborative relationship, which
is regarded as a new kind of knowledge for low-level vision tasks. For kernel
decomposition, we apply the depth-wise separation of convolutional kernels to
build a light-weighted Convolutional Neural Network (CNN) from the original
network. Extensive experiments show the effectiveness of the compression method
when applied to the state-of-the-art facial makeup transfer network --
BeautyGAN.

本研究旨在压缩面部化妆转换网络，通过协作蒸馏和核分解这两种方法进行模型压缩，从而提高模型运行效率，试验结果表明，这种压缩方法可以有效地应用于美妆转换领域。