In this paper, we present a novel approach for incorporating external
knowledge in Recurrent Neural Networks (RNNs). We propose the integration of
lexicon features into the self-attention mechanism of RNN-based architectures.
This form of conditioning on the attention distribution, enforces the
contribution of the most salient words for the task at hand. We introduce three
methods, namely attentional concatenation, feature-based gating and affine
transformation. Experiments on six benchmark datasets show the effectiveness of
our methods. Attentional feature-based gating yields consistent performance
improvement across tasks. Our approach is implemented as a simple add-on module
for RNN-based models with minimal computational overhead and can be adapted to
any deep neural architecture.

本文提出了一种新颖的方法，将外部知识纳入循环神经网络中，通过在注意力分布中引入词汇特征来提高模型的表现，我们介绍了三种方法，并在六个基准数据集上进行了实验，结果表明这种方法可以产生稳定的性能提升，并且可以适用于任何深度神经网络架构。