Recent advances in machine learning theory showed that interpolation to noisy
samples using over-parameterized machine learning algorithms always leads to
inconsistency. However, this work surprisingly discovers that interpolated
machine learning can exhibit benign overfitting and consistency when using
physics-informed learning for supervised tasks governed by partial differential
equations (PDEs) describing laws of physics. An analysis provides an asymptotic
Sobolev norm learning curve for kernel ridge(less) regression addressing linear
inverse problems involving elliptic PDEs. The results reveal that the PDE
operators can stabilize variance and lead to benign overfitting for
fixed-dimensional problems, contrasting standard regression settings. The
impact of various inductive biases introduced by minimizing different Sobolev
norms as implicit regularization is also examined. Notably, the convergence
rate is independent of the specific (smooth) inductive bias for both ridge and
ridgeless regression. For regularized least squares estimators, all (smooth
enough) inductive biases can achieve optimal convergence rates when the
regularization parameter is properly chosen. The smoothness requirement
recovers a condition previously found in the Bayesian setting and extends
conclusions to minimum norm interpolation estimators.

最近机器学习理论的进展表明，使用过参数化的机器学习算法对带有噪声样本进行插值总是会导致不一致性。然而，这项工作却令人惊讶地发现，在描述物理定律的偏微分方程（PDEs）控制的监督任务中，使用物理知识驱动的学习进行插值的机器学习可以表现出良性过拟合和一致性。分析了解决涉及椭圆型 PDE 的线性逆问题的核岭（无岭）回归的渐进 Sobolev 范数学习曲线。结果显示，PDE 算子可以稳定方差并导致固定维度问题的良性过拟合，与标准回归设置形成对比。还研究了通过最小化不同 Sobolev 范数引入的各种归纳偏差作为隐式正则化的影响。值得注意的是，对于岭回归和无岭回归，收敛速度与具体的（平滑）归纳偏差无关。对于正则化最小二乘估计器，当适当地选择正则化参数时，所有（足够平滑的）归纳偏差都可以实现最优的收敛速度。平滑性要求恢复了先前在贝叶斯设置中发现的条件，并将结论扩展到最小范数插值估计器。