The goal of this paper is to revisit Kernel Principal Component Analysis
(KPCA) through dualization of a difference of convex functions. This allows to
naturally extend KPCA to multiple objective functions and leads to efficient
gradient-based algorithms avoiding the expensive SVD of the Gram matrix.
Particularly, we consider objective functions that can be written as Moreau
envelopes, demonstrating how to promote robustness and sparsity within the same
framework. The proposed method is evaluated on synthetic and real-world
benchmarks, showing significant speedup in KPCA training time as well as
highlighting the benefits in terms of robustness and sparsity.

本文通过 DC 函数的对偶化重新审视核主成分分析（KPCA），从而将 KPCA 自然地扩展到多个目标函数，并导致避免 Gram 矩阵的昂贵 SVD 的高效梯度下降算法。特别地，我们考虑可以写成 Moreau 套外壳的目标函数，展示了如何在同一框架中促进鲁棒性和稀疏性。提出的方法在合成和实际基准测试中进行了评估，显示出 KPCA 训练时间的显着加速以及在鲁棒性和稀疏性方面的好处。

通过对偶扩展核 PCA：稀疏性，鲁棒性和快速算法

Extending Kernel PCA through Dualization: Sparsity, Robustness and Fast  Algorithms

Federated learning (FL) is a decentralized and privacy-preserving machine
learning technique in which a group of clients collaborate with a server to
learn a global model without sharing clients' data. One challenge associated
with FL is statistical diversity among clients, which restricts the global
model from delivering good performance on each client's task. To address this,
we propose an algorithm for personalized FL (pFedMe) using Moreau envelopes as
clients' regularized loss functions, which help decouple personalized model
optimization from the global model learning in a bi-level problem stylized for
personalized FL. Theoretically, we show that pFedMe's convergence rate is
state-of-the-art: achieving quadratic speedup for strongly convex and sublinear
speedup of order 2/3 for smooth nonconvex objectives. Experimentally, we verify
that pFedMe excels at empirical performance compared with the vanilla FedAvg
and Per-FedAvg, a meta-learning based personalized FL algorithm.

该研究提出了一种基于 Moreau envelopes 的个性化联邦学习算法 (pFedMe)，用于解决客户端之间的统计差异性问题并实现个性化模型优化和全局模型学习的分离，该算法在理论上已证明具有先进的收敛速度，在实验中也证明在表现上优于 vanilla FedAvg 和 Per-FedAvg 等算法。