Regularization plays a pivotal role in ill-posed machine learning and inverse
problems. However, the fundamental comparative analysis of various
regularization norms remains open. We establish a small noise analysis
framework to assess the effects of norms in Tikhonov and RKHS regularizations,
in the context of ill-posed linear inverse problems with Gaussian noise. This
framework studies the convergence rates of regularized estimators in the small
noise limit and reveals the potential instability of the conventional
L2-regularizer. We solve such instability by proposing an innovative class of
adaptive fractional RKHS regularizers, which covers the L2 Tikhonov and RKHS
regularizations by adjusting the fractional smoothness parameter. A surprising
insight is that over-smoothing via these fractional RKHSs consistently yields
optimal convergence rates, but the optimal hyper-parameter may decay too fast
to be selected in practice.

本文通过建立一个小噪声分析框架，比较了 Tikhonov 和 RKHS 正则化问题中各种正则化规范的效果，发现 L2 正则化器存在潜在的不稳定性，并通过提出一种新颖的自适应分数 RKHS 正则化器来解决其不稳定性，实验结果表明，过度平滑可以提高收敛速度，但最优超参数难以实际选择。