Several recent works find empirically that the average test error of deep
neural networks can be estimated via the prediction disagreement of models,
which does not require labels. In particular, Jiang et al. (2022) show for the
disagreement between two separately trained networks that this `Generalization
Disagreement Equality' follows from the well-calibrated nature of deep
ensembles under the notion of a proposed `class-aggregated calibration.' In
this reproduction, we show that the suggested theory might be impractical
because a deep ensemble's calibration can deteriorate as prediction
disagreement increases, which is precisely when the coupling of test error and
disagreement is of interest, while labels are needed to estimate the
calibration on new datasets. Further, we simplify the theoretical statements
and proofs, showing them to be straightforward within a probabilistic context,
unlike the original hypothesis space view employed by Jiang et al. (2022).

本文介绍了深度神经网络的预测误差可以通过模型的预测不一致性进行估计，其中，“广义差异等式” 遵循了深度集合的良好校准性，并提出了 “类别聚合校准” 的概念。但我们发现该理论可能不实用，因为预测不一致性增加时，深度集合的校准性可能会降低，并需要标签来估计新数据集上的校准性。我们简化了理论陈述和证明，并在概率上下文中展示它们是简单明了的。