Random feature (RF) has been widely used for node consistency in
decentralized kernel ridge regression (KRR). Currently, the consistency is
guaranteed by imposing constraints on coefficients of features, necessitating
that the random features on different nodes are identical. However, in many
applications, data on different nodes varies significantly on the number or
distribution, which calls for adaptive and data-dependent methods that generate
different RFs. To tackle the essential difficulty, we propose a new
decentralized KRR algorithm that pursues consensus on decision functions, which
allows great flexibility and well adapts data on nodes. The convergence is
rigorously given and the effectiveness is numerically verified: by capturing
the characteristics of the data on each node, while maintaining the same
communication costs as other methods, we achieved an average regression
accuracy improvement of 25.5\% across six real-world data sets.

在去中心化的核岭回归中，为了保证节点之间的一致性，通常会对特征系数施加约束，但是在许多应用中，不同节点上的数据在数量或分布上存在显著差异，因此需要能够生成不同随机特征的自适应和数据相关方法。针对这个关键难题，本文提出了一种新的去中心化核岭回归算法，该算法通过追求决策函数的一致性，实现了对节点上数据的灵活适应。经过严格的收敛性分析和数值验证，我们得出结论：在保持与其他方法相同的通信开销的同时，我们在六个真实世界数据集中平均提高了 25.5％的回归准确性。

基于数据相关随机特征的去中心化核岭回归

Decentralized Kernel Ridge Regression Based on Data-dependent Random  Feature

Random Feature (RF) models are used as efficient parametric approximations of
kernel methods. We investigate, by means of random matrix theory, the
connection between Gaussian RF models and Kernel Ridge Regression (KRR). For a
Gaussian RF model with $P$ features, $N$ data points, and a ridge $\lambda$, we
show that the average (i.e. expected) RF predictor is close to a KRR predictor
with an effective ridge $\tilde{\lambda}$. We show that $\tilde{\lambda} >
\lambda$ and $\tilde{\lambda} \searrow \lambda$ monotonically as $P$ grows,
thus revealing the implicit regularization effect of finite RF sampling. We
then compare the risk (i.e. test error) of the $\tilde{\lambda}$-KRR predictor
with the average risk of the $\lambda$-RF predictor and obtain a precise and
explicit bound on their difference. Finally, we empirically find an extremely
good agreement between the test errors of the average $\lambda$-RF predictor
and $\tilde{\lambda}$-KRR predictor.

本文探讨了随机特征模型和核岭回归之间的联系，并发现了有限 RF 取样的隐式正则化效应，同时对比了使用 KRR 预测器的风险和使用 RF 预测器的平均风险并获得了它们之间差异的明确界限，最终在实验中发现了平均 λ-RF 预测和 tilde λ-KRR 预测器之间的极好一致性。