Ensembles of separate neural networks (NNs) have shown superior accuracy and
confidence calibration over single NN across tasks. Recent methods compress
ensembles within a single network via early exits or multi-input multi-output
frameworks. However, the landscape of these methods is fragmented thus far,
making it difficult to choose the right approach for a given task. Furthermore,
the algorithmic performance of these methods is behind the ensemble of separate
NNs and requires extensive architecture tuning. We propose a novel methodology
unifying these approaches into a Single Architecture Ensemble (SAE). Our method
learns the optimal number and depth of exits per ensemble input in a single NN.
This enables the SAE framework to flexibly tailor its configuration for a given
architecture or application. We evaluate SAEs on image classification and
regression across various network architecture types and sizes. We demonstrate
competitive accuracy or confidence calibration to baselines while reducing the
compute operations or parameter count by up to $1.5{\sim}3.7\times$.

通过单个神经网络中的早期退出和多输入多输出框架，将分散的神经网络集成为一个单一的体系结构集合，提高准确度和置信度校准，并减少运算和参数量。