Scaling machine learning models significantly improves their performance.
However, such gains come at the cost of inference being slow and
resource-intensive. Early-exit neural networks (EENNs) offer a promising
solution: they accelerate inference by allowing intermediate layers to exit and
produce a prediction early. Yet a fundamental issue with EENNs is how to
determine when to exit without severely degrading performance. In other words,
when is it 'safe' for an EENN to go 'fast'? To address this issue, we
investigate how to adapt frameworks of risk control to EENNs. Risk control
offers a distribution-free, post-hoc solution that tunes the EENN's exiting
mechanism so that exits only occur when the output is of sufficient quality. We
empirically validate our insights on a range of vision and language tasks,
demonstrating that risk control can produce substantial computational savings,
all the while preserving user-specified performance goals.

通过应用风险控制机制，我们实现了针对早期停止神经网络（EENNs）的退出机制，在保持用户指定的性能目标的同时，大幅减少计算资源的使用。

快速而安全：带有风险控制的提前退出

Fast yet Safe: Early-Exiting with Risk Control

This article summarizes principles and ideas from the emerging area of
applying \textit{conditional computation} methods to the design of neural
networks. In particular, we focus on neural networks that can dynamically
activate or de-activate parts of their computational graph conditionally on
their input. Examples include the dynamic selection of, e.g., input tokens,
layers (or sets of layers), and sub-modules inside each layer (e.g., channels
in a convolutional filter). We first provide a general formalism to describe
these techniques in an uniform way. Then, we introduce three notable
implementations of these principles: mixture-of-experts (MoEs) networks, token
selection mechanisms, and early-exit neural networks. The paper aims to provide
a tutorial-like introduction to this growing field. To this end, we analyze the
benefits of these modular designs in terms of efficiency, explainability, and
transfer learning, with a focus on emerging applicative areas ranging from
automated scientific discovery to semantic communication.

这篇论文总结了将条件计算方法应用于神经网络设计的新兴领域的原理和思想，特别关注可以根据输入动态激活或停用计算图的部分的神经网络。论文介绍了三种显著的实现方法：专家混合网络，令牌选择机制和提前终止神经网络，并分析了这些模块化设计在效率、可解释性和迁移学习方面的优点，重点关注从自动科学发现到语义通信等新兴应用领域。

神经网络中的条件计算：原理与研究趋势

Conditional computation in neural networks: principles and research  trends

Early-exit neural networks (EENNs) facilitate adaptive inference by producing
predictions at multiple stages of the forward pass. In safety-critical
applications, these predictions are only meaningful when complemented with
reliable uncertainty estimates. Yet, due to their sequential structure, an
EENN's uncertainty estimates should also be consistent: labels that are deemed
improbable at one exit should not reappear within the confidence interval / set
of later exits. We show that standard uncertainty quantification techniques,
like Bayesian methods or conformal prediction, can lead to inconsistency across
exits. We address this problem by applying anytime-valid confidence sequences
(AVCSs) to the exits of EENNs. By design, AVCSs maintain consistency across
exits. We examine the theoretical and practical challenges of applying AVCSs to
EENNs and empirically validate our approach on both regression and
classification tasks.

早期退出神经网络（EENN）采用随时有效的置信序列（AVCSs）以解决不确定性估计中的一致性问题，并在回归和分类任务中进行实证验证。

早停神经网络的一致不确定性评估的随时有效置信区间

Anytime-Valid Confidence Sequences for Consistent Uncertainty Estimation  in Early-Exit Neural Networks

Modern predictive models are often deployed to environments in which
computational budgets are dynamic. Anytime algorithms are well-suited to such
environments as, at any point during computation, they can output a prediction
whose quality is a function of computation time. Early-exit neural networks
have garnered attention in the context of anytime computation due to their
capability to provide intermediate predictions at various stages throughout the
network. However, we demonstrate that current early-exit networks are not
directly applicable to anytime settings, as the quality of predictions for
individual data points is not guaranteed to improve with longer computation. To
address this shortcoming, we propose an elegant post-hoc modification, based on
the Product-of-Experts, that encourages an early-exit network to become
gradually confident. This gives our deep models the property of conditional
monotonicity in the prediction quality -- an essential stepping stone towards
truly anytime predictive modeling using early-exit architectures. Our empirical
results on standard image-classification tasks demonstrate that such behaviors
can be achieved while preserving competitive accuracy on average.

本研究提出了一种基于 Product-of-Experts 的修改方法，使得早期退出网络在计算过程中逐渐变得自信，并具有预测质量的条件单调性，从而在保持竞争力的前提下实现了真正的任意时间预测建模。