When artificial neural networks have demonstrated exceptional practical
success in a variety of domains, investigations into their theoretical
characteristics, such as their approximation power, statistical properties, and
generalization performance, have made significant strides. In this paper, we
construct a novel theory for understanding the effectiveness of neural networks
by discovering the mystery underlying a common practice during neural network
model construction: sample splitting. Our theory demonstrates that, the optimal
hyperparameters derived from sample splitting can enable a neural network model
that asymptotically minimizes the prediction risk. We conduct extensive
experiments across different application scenarios and network architectures,
and the results manifest our theory's effectiveness.

本文针对神经网络在实际应用中的表现提出了一个新理论，通过发现在神经网络模型构建过程中的一个常见做法 —— 样本分裂的奥秘来探究其有效性，该理论表明，从样本分裂中得出的最佳超参数能够使神经网络模型渐近地最小化预测风险。我们在不同的应用场景和网络架构中进行了大量实验，结果证明了我们理论的有效性。

朝着最佳神经网络：样本拆分在超参数选择中的作用

Towards Optimal Neural Networks: the Role of Sample Splitting in  Hyperparameter Selection

We prove calibration guarantees for the popular histogram binning (also
called uniform-mass binning) method of Zadrozny and Elkan [2001]. Histogram
binning has displayed strong practical performance, but theoretical guarantees
have only been shown for sample split versions that avoid 'double dipping' the
data. We demonstrate that the statistical cost of sample splitting is
practically significant on a credit default dataset. We then prove calibration
guarantees for the original method that double dips the data, using a certain
Markov property of order statistics. Based on our results, we make practical
recommendations for choosing the number of bins in histogram binning. In our
illustrative simulations, we propose a new tool for assessing calibration --
validity plots -- which provide more information than an ECE estimate. Code for
this work will be made publicly available at
this https URL

本文提出了针对 Zadrozny 和 Elkan [2001] 的常用的直方图分箱（也称为均匀质量分箱）方法的校准保证。作者分析了双倍数据的原始方法的实用性，并使用一定的统计学性质证明了直方图分箱方法的校准保证。作者提出了新的工具 - validity plots，用于评估校准并提出了实用建议。