While much progress has been made in how to encode a text sequence into a
sequence of vectors, less attention has been paid to how to aggregate these
preceding vectors (outputs of RNN/CNN) into fixed-size encoding vector.
Usually, a simple max or average pooling is used, which is a bottom-up and
passive way of aggregation and lack of guidance by task information. In this
paper, we propose an aggregation mechanism to obtain a fixed-size encoding with
a dynamic routing policy. The dynamic routing policy is dynamically deciding
that what and how much information need be transferred from each word to the
final encoding of the text sequence. Following the work of Capsule Network, we
design two dynamic routing policies to aggregate the outputs of RNN/CNN
encoding layer into a final encoding vector. Compared to the other aggregation
methods, dynamic routing can refine the messages according to the state of
final encoding vector. Experimental results on five text classification tasks
show that our method outperforms other aggregating models by a significant
margin. Related source code is released on our github page.

本文提出一种聚合机制来获取具有动态路由策略的固定大小的向量表示文本序列，并通过实验结果在五个文本分类任务中展现了其优越性。

动态路由的序列编码信息聚合

Information Aggregation via Dynamic Routing for Sequence Encoding

In a variety of application domains the content to be recommended to users is
associated with text. This includes research papers, movies with associated
plot summaries, news articles, blog posts, etc. Recommendation approaches based
on latent factor models can be extended naturally to leverage text by employing
an explicit mapping from text to factors. This enables recommendations for new,
unseen content, and may generalize better, since the factors for all items are
produced by a compactly-parametrized model. Previous work has used topic models
or averages of word embeddings for this mapping. In this paper we present a
method leveraging deep recurrent neural networks to encode the text sequence
into a latent vector, specifically gated recurrent units (GRUs) trained
end-to-end on the collaborative filtering task. For the task of scientific
paper recommendation, this yields models with significantly higher accuracy. In
cold-start scenarios, we beat the previous state-of-the-art, all of which
ignore word order. Performance is further improved by multi-task learning,
where the text encoder network is trained for a combination of content
recommendation and item metadata prediction. This regularizes the collaborative
filtering model, ameliorating the problem of sparsity of the observed rating
matrix.

本文介绍了一种基于深度递归神经网络的内容推荐方法，通过将文本序列映射至潜在向量实现推荐，在科学论文推荐任务中，该方法精度显著优于传统基于主题模型或词向量平均值的推荐方法。此外，使用多任务学习进一步提高了推荐精度。