This paper explores a novel dynamic network for vision and language tasks,
where the inferring structure is customized on the fly for different inputs.
Most previous state-of-the-art approaches are static and hand-crafted networks,
which not only heavily rely on expert knowledge, but also ignore the semantic
diversity of input samples, therefore resulting in suboptimal performance. To
address these issues, we propose a novel Dynamic Transformer Network (DTNet)
for image captioning, which dynamically assigns customized paths to different
samples, leading to discriminative yet accurate captions. Specifically, to
build a rich routing space and improve routing efficiency, we introduce five
types of basic cells and group them into two separate routing spaces according
to their operating domains, i.e., spatial and channel. Then, we design a
Spatial-Channel Joint Router (SCJR), which endows the model with the capability
of path customization based on both spatial and channel information of the
input sample. To validate the effectiveness of our proposed DTNet, we conduct
extensive experiments on the MS-COCO dataset and achieve new state-of-the-art
performance on both the Karpathy split and the online test server.

本文探索一种新颖的动态网络以应对视觉和语言任务，其中推理结构针对不同输入动态定制。通过引入基本单元并在空间和通道运算领域分组，以构建丰富的路径空间和提升路径选择效率，我们设计了一个空间 - 通道联合路由器来根据输入样本的空间和通道信息进行路径定制，并在 MS-COCO 数据集上进行实验证明了提出的动态变压器网络的有效性，获得了 Karpathy 分割和在线测试服务器上的最新最佳性能。