With the promising progress of deep neural networks, layer aggregation has
been used to fuse information across layers in various fields, such as computer
vision and machine translation. However, most of the previous methods combine
layers in a static fashion in that their aggregation strategy is independent of
specific hidden states. Inspired by recent progress on capsule networks, in
this paper we propose to use routing-by-agreement strategies to aggregate
layers dynamically. Specifically, the algorithm learns the probability of a
part (individual layer representations) assigned to a whole (aggregated
representations) in an iterative way and combines parts accordingly. We
implement our algorithm on top of the state-of-the-art neural machine
translation model TRANSFORMER and conduct experiments on the widely-used WMT14
English-German and WMT17 Chinese-English translation datasets. Experimental
results across language pairs show that the proposed approach consistently
outperforms the strong baseline model and a representative static aggregation
model.

本文提出一种采用路由 - 接受策略来动态聚合层信息的方法，将其实现于 Transformer 神经机器翻译模型之上，并在常用数据集上测试，结果表明该方法在多语言翻译上表现优异。

基于共识路由的神经机器翻译动态层聚合

Dynamic Layer Aggregation for Neural Machine Translation with  Routing-by-Agreement

Advanced neural machine translation (NMT) models generally implement encoder
and decoder as multiple layers, which allows systems to model complex functions
and capture complicated linguistic structures. However, only the top layers of
encoder and decoder are leveraged in the subsequent process, which misses the
opportunity to exploit the useful information embedded in other layers. In this
work, we propose to simultaneously expose all of these signals with layer
aggregation and multi-layer attention mechanisms. In addition, we introduce an
auxiliary regularization term to encourage different layers to capture diverse
information. Experimental results on widely-used WMT14 English-German and WMT17
Chinese-English translation data demonstrate the effectiveness and universality
of the proposed approach.

本文提出了一种新的神经机器翻译方法，采用层聚合和多层注意力机制，通过引入辅助正则化项促进不同层捕获多样化信息，实验结果表明该方法在 WMT14 英德和 WMT17 中英数据上具有普适性与有效性。