Dynamic computation has emerged as a promising avenue to enhance the
inference efficiency of deep networks. It allows selective activation of
computational units, leading to a reduction in unnecessary computations for
each input sample. However, the actual efficiency of these dynamic models can
deviate from theoretical predictions. This mismatch arises from: 1) the lack of
a unified approach due to fragmented research; 2) the focus on algorithm design
over critical scheduling strategies, especially in CUDA-enabled GPU contexts;
and 3) challenges in measuring practical latency, given that most libraries
cater to static operations. Addressing these issues, we unveil the
Latency-Aware Unified Dynamic Networks (LAUDNet), a framework that integrates
three primary dynamic paradigms-spatially adaptive computation, dynamic layer
skipping, and dynamic channel skipping. To bridge the theoretical and practical
efficiency gap, LAUDNet merges algorithmic design with scheduling optimization,
guided by a latency predictor that accurately gauges dynamic operator latency.
We've tested LAUDNet across multiple vision tasks, demonstrating its capacity
to notably reduce the latency of models like ResNet-101 by over 50% on
platforms such as V100, RTX3090, and TX2 GPUs. Notably, LAUDNet stands out in
balancing accuracy and efficiency. Code is available at:
this https URL

动态计算已成为提高深度网络推理效率的一种有前途的方法，我们提出了一个名为 “LAUDNet” 的框架，它集成了三个主要的动态范例，以缩短模型的延迟，通过算法设计与调度优化相结合，准确衡量动态操作延迟的潜在预测器指导。我们在多个视觉任务上测试了 LAUDNet，其在 V100、RTX3090 和 TX2 GPU 上能够显著降低 ResNet-101 等模型的延迟超过 50％，而且在准确性和效率之间取得良好的平衡。

面向高效图像识别的低延迟统一动态网络

Latency-aware Unified Dynamic Networks for Efficient Image Recognition

When solving a problem, human beings have the adaptive ability in terms of
the type of information they use, the procedure they take, and the amount of
time they spend approaching and solving the problem. However, most standard
neural networks have the same function type and fixed computation budget on
different samples regardless of their nature and difficulty. Adaptivity is a
powerful paradigm as it not only imbues practitioners with flexibility
pertaining to the downstream usage of these models but can also serve as a
powerful inductive bias for solving certain challenging classes of problems. In
this work, we propose a new strategy, AdaTape, that enables dynamic computation
in neural networks via adaptive tape tokens. AdaTape employs an elastic input
sequence by equipping an existing architecture with a dynamic read-and-write
tape. Specifically, we adaptively generate input sequences using tape tokens
obtained from a tape bank that can either be trainable or generated from input
data. We analyze the challenges and requirements to obtain dynamic sequence
content and length, and propose the Adaptive Tape Reader (ATR) algorithm to
achieve both objectives. Via extensive experiments on image recognition tasks,
we show that AdaTape can achieve better performance while maintaining the
computational cost.

本研究通过使用 “tape token” 实现了神经网络的动态计算，通过对图像识别任务的实验表明了这种方法的有效性。