Federated Learning with Model Distillation (FedMD) is a nascent collaborative learning paradigm, where only output logits of public datasets are transmitted as distilled knowledge, instead of passing on private model parameters that are susceptible to gradient inversion attacks, a known privacy risk in federated learning. In this paper, we found that even though sharing output logits of public datasets is safer than directly sharing gradients, there still exists a substantial risk of data exposure caused by carefully designed malicious attacks. Our study shows that a malicious server can inject a PLI (Paired-Logits Inversion) attack against FedMD and its variants by training an inversion neural network that exploits the confidence gap between the server and client models. Experiments on multiple facial recognition datasets validate that under FedMD-like schemes, by using paired server-client logits of public datasets only, the malicious server is able to reconstruct private images on all tested benchmarks with a high success rate.

本文介绍了一种名为FedMD的联合学习方案，使用模型蒸馏来保证隐私性，而不是直接共享模型参数。然而，我们发现即使是共享公共数据集的输出日志比直接共享渐变更为安全，但仍然存在数据曝光的风险。作者研究表明，恶意服务器可以利用PLI攻击FedMD及其变体。在多个面部识别数据集上的实验验证了作者发现的成果。

Paired-Logits 逆向攻击恢复图像的 FedMD 破解