Machine learning models can solve complex tasks but often require significant
computational resources during inference. This has led to the development of
various post-training computation reduction methods that tackle this issue in
different ways, such as quantization which reduces the precision of weights and
arithmetic operations, and dynamic networks which adapt computation to the
sample at hand. In this work, we propose a more general dynamic network that
can combine both quantization and early exit dynamic network: QuEE. Our
algorithm can be seen as a form of soft early exiting or input-dependent
compression. Rather than a binary decision between exiting or continuing, we
introduce the possibility of continuing with reduced computation. This
complicates the traditionally considered early exiting problem, which we solve
through a principled formulation. The crucial factor of our approach is
accurate prediction of the potential accuracy improvement achievable through
further computation. We demonstrate the effectiveness of our method through
empirical evaluation, as well as exploring the conditions for its success on 4
classification datasets.

提出了一种结合量化和早期退出动态网络的更一般的动态网络方法 QuEE，通过准确预测进一步计算可能获得的潜在准确性改进，解决了传统的早期退出问题。