A central problem in data science is to use potentially noisy samples of an unknown function to predict function values for unseen inputs. In classical statistics, the predictive error is understood as a trade-off between the bias and the variance that balances model simplicity with its ability to fit complex functions. However, over-parameterized models exhibit counter-intuitive behaviors, such as "double descent" in which models of increasing complexity exhibit decreasing generalization error. We introduce an alternative paradigm called the generalized aliasing decomposition. We explain the asymptotically small error of complex models as a systematic "de-aliasing" that occurs in the over-parameterized regime. In the limit of large models, the contribution due to aliasing vanishes, leaving an expression for the asymptotic total error we call the invertibility failure of very large models on few training points. Because the generalized aliasing decomposition can be explicitly calculated from the relationship between model class and samples without seeing any data labels, it can answer questions related to experimental design and model selection before collecting data or performing experiments. We demonstrate this approach using several examples, including classical regression problems and a cluster expansion model used in materials science.

本研究解决了数据科学中使用潜在噪声样本预测未知函数值的核心问题。提出的广义别名分解新方法揭示了复杂模型的渐近小误差是由于在过参数化状态下进行的系统“去别名”。研究表明，在训练点数据较少的情况下，极大模型的可逆性失败导致的渐近总误差的明确计算，能够在收集数据前为实验设计和模型选择提供指导。

风险的别名和无标签分解：超越偏差-方差权衡