Ridgeless regression has garnered attention among researchers, particularly
in light of the ``Benign Overfitting'' phenomenon, where models interpolating
noisy samples demonstrate robust generalization. However, kernel ridgeless
regression does not always perform well due to the lack of flexibility. This
paper enhances kernel ridgeless regression with Locally-Adaptive-Bandwidths
(LAB) RBF kernels, incorporating kernel learning techniques to improve
performance in both experiments and theory. For the first time, we demonstrate
that functions learned from LAB RBF kernels belong to an integral space of
Reproducible Kernel Hilbert Spaces (RKHSs). Despite the absence of explicit
regularization in the proposed model, its optimization is equivalent to solving
an $\ell_0$-regularized problem in the integral space of RKHSs, elucidating the
origin of its generalization ability. Taking an approximation analysis
viewpoint, we introduce an $l_q$-norm analysis technique (with $0<q<1$) to
derive the learning rate for the proposed model under mild conditions. This
result deepens our theoretical understanding, explaining that our algorithm's
robust approximation ability arises from the large capacity of the integral
space of RKHSs, while its generalization ability is ensured by sparsity,
controlled by the number of support vectors. Experimental results on both
synthetic and real datasets validate our theoretical conclusions.

本文通过加入局部自适应带宽的 RBF 核和核学习技术，提升了核无岭回归，并首次证明了由 LAB RBF 核学习的函数属于可重构核希尔伯特空间。尽管所提出模型中没有显式正则化，但其优化等效于在可重构核希尔伯特空间中解决一个 l0 正则化问题，解释了其泛化能力的来源。在近似分析角度下，我们引入了一种 lq - 范数分析技术（其中 0<q<1），以在适度条件下推导所提模型的学习率。实验结果在合成和真实数据集上验证了我们的理论推论。