Cascades are a classical strategy to enable inference cost to vary adaptively
across samples, wherein a sequence of classifiers are invoked in turn. A
deferral rule determines whether to invoke the next classifier in the sequence,
or to terminate prediction. One simple deferral rule employs the confidence of
the current classifier, e.g., based on the maximum predicted softmax
probability. Despite being oblivious to the structure of the cascade -- e.g.,
not modelling the errors of downstream models -- such confidence-based deferral
often works remarkably well in practice. In this paper, we seek to better
understand the conditions under which confidence-based deferral may fail, and
when alternate deferral strategies can perform better. We first present a
theoretical characterisation of the optimal deferral rule, which precisely
characterises settings under which confidence-based deferral may suffer. We
then study post-hoc deferral mechanisms, and demonstrate they can significantly
improve upon confidence-based deferral in settings where (i) downstream models
are specialists that only work well on a subset of inputs, (ii) samples are
subject to label noise, and (iii) there is distribution shift between the train
and test set.

通过理论分析和实验证明在级联模型中，基于置信度的预测推迟机制可能会在下游模型专业性、标签噪声和训练集与测试集分布偏移等情况下失效，进而提出后处理推迟机制并展示了其在这些情况下能够显著提升性能。