Training a deep neural network often involves stochastic optimization,
meaning each run will produce a different model. The seed used to initialize
random elements of the optimization procedure heavily influences the quality of
a trained model, which may be obscure from many commonly reported summary
statistics, like accuracy. However, random seed is often not included in
hyper-parameter optimization, perhaps because the relationship between seed and
model quality is hard to describe. This work attempts to describe the
relationship between deep net models trained with different random seeds and
the behavior of the expected model. We adopt robust hypothesis testing to
propose a novel summary statistic for network similarity, referred to as the
$\alpha$-trimming level. We use the $\alpha$-trimming level to show that the
empirical cumulative distribution function of an ensemble model created from a
collection of trained models with different random seeds approximates the
average of these functions as the number of models in the collection grows
large. This insight provides guidance for how many random seeds should be
sampled to ensure that an ensemble of these trained models is a reliable
representative. We also show that the $\alpha$-trimming level is more
expressive than different performance metrics like validation accuracy, churn,
or expected calibration error when taken alone and may help with random seed
selection in a more principled fashion. We demonstrate the value of the
proposed statistic in real experiments and illustrate the advantage of
fine-tuning over random seed with an experiment in transfer learning.

深度神经网络的训练通常涉及随机优化，种子的选择对于训练模型的质量至关重要，本文通过鲁棒的假设检验提出了一个新的网络相似性的统计量，利用该统计量来指导随机种子的选择，同时展示了该统计量在实验中的价值和在迁移学习中调优相比随机种子选取的优势。

使用鲁棒非参数统计方法测量模型变异性

Measuring model variability using robust non-parametric testing

Approximate Bayesian Computation (ABC) methods are used to approximate
posterior distributions in models with unknown or computationally intractable
likelihoods. Both the accuracy and computational efficiency of ABC depend on
the choice of summary statistic, but outside of special cases where the optimal
summary statistics are known, it is unclear which guiding principles can be
used to construct effective summary statistics. In this paper we explore the
possibility of automating the process of constructing summary statistics by
training deep neural networks to predict the parameters from artificially
generated data: the resulting summary statistics are approximately posterior
means of the parameters. With minimal model-specific tuning, our method
constructs summary statistics for the Ising model and the moving-average model,
which match or exceed theoretically-motivated summary statistics in terms of
the accuracies of the resulting posteriors.

通过训练深度神经网络，本研究旨在构建可靠的摘要统计信息，以近似表示参数的后验平均值，以提高 ABC 方法在模型参数推断中的精度和效率。在 Ising 模型和移动平均模型中，我们的方法构建的摘要统计量在精度上不仅匹配了理论上的最优摘要统计量，而且超出了其精度。