Recently, the tokens of images share the same static data flow in many dense
networks. However, challenges arise from the variance among the objects in
images, such as large variations in the spatial scale and difficulties of
recognition for visual entities. In this paper, we propose a data-dependent
token routing strategy to elaborate the routing paths of image tokens for
Dynamic Vision Transformer, dubbed DiT. The proposed framework generates a
data-dependent path per token, adapting to the object scales and visual
discrimination of tokens. In feed-forward, the differentiable routing gates are
designed to select the scaling paths and feature transformation paths for image
tokens, leading to multi-path feature propagation. In this way, the impact of
object scales and visual discrimination of image representation can be
carefully tuned. Moreover, the computational cost can be further reduced by
giving budget constraints to the routing gate and early-stopping of feature
extraction. In experiments, our DiT achieves superior performance and favorable
complexity/accuracy trade-offs than many SoTA methods on ImageNet
classification, object detection, instance segmentation, and semantic
segmentation. Particularly, the DiT-B5 obtains 84.8\% top-1 Acc on ImageNet
with 10.3 GFLOPs, which is 1.0\% higher than that of the SoTA method with
similar computational complexity. These extensive results demonstrate that DiT
can serve as versatile backbones for various vision tasks.

提出了一种数据相关的令牌路由策略用于图像令牌的动态视觉转换器 (DiT)，以适应对象尺度和视觉识别的变化，并通过选择多路径特征传播来精心调整图像表达的对象尺度和视觉识别的影响，从而实现更好的性能和良好的复杂度 / 准确度平衡。