We study theoretical properties of a broad class of regularized algorithms with vector-valued output. These spectral algorithms include kernel ridge regression, kernel principal component regression, various implementations of gradient descent and many more. Our contributions are twofold. First, we rigorously confirm the so-called saturation effect for ridge regression with vector-valued output by deriving a novel lower bound on learning rates; this bound is shown to be suboptimal when the smoothness of the regression function exceeds a certain level. Second, we present the upper bound for the finite sample risk general vector-valued spectral algorithms, applicable to both well-specified and misspecified scenarios (where the true regression function lies outside of the hypothesis space) which is minimax optimal in various regimes. All of our results explicitly allow the case of infinite-dimensional output variables, proving consistency of recent practical applications.

我们研究具有向量值输出的一类正则化算法的理论性质，包括核岭回归、核主成分回归、梯度下降的各种实现等。我们的贡献有两个方面：首先，通过推导出一种新的学习速率下限，我们严格确认了具有向量值输出的岭回归的所谓饱和效应；当回归函数的平滑度超过一定水平时，该下限被证明是次优的。其次，我们给出了适用于有限样本风险的一般向量值谱算法的上限，适用于规范良好和规范错误的情况（真实的回归函数在假设空间之外），在多种情况下都是最小化最优的。我们的所有结果明确允许无限维度的输出变量情况，证明了最近实际应用的一致性。

向量值谱正则化学习算法的最优速率