Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely
used on NLP tasks to capture the long-term and local dependencies,
respectively. Attention mechanisms have recently attracted enormous interest
due to their highly parallelizable computation, significantly less training
time, and flexibility in modeling dependencies. We propose a novel attention
mechanism in which the attention between elements from input sequence(s) is
directional and multi-dimensional (i.e., feature-wise). A light-weight neural
net, "Directional Self-Attention Network (DiSAN)", is then proposed to learn
sentence embedding, based solely on the proposed attention without any RNN/CNN
structure. DiSAN is only composed of a directional self-attention with temporal
order encoded, followed by a multi-dimensional attention that compresses the
sequence into a vector representation. Despite its simple form, DiSAN
outperforms complicated RNN models on both prediction quality and time
efficiency. It achieves the best test accuracy among all sentence encoding
methods and improves the most recent best result by 1.02% on the Stanford
Natural Language Inference (SNLI) dataset, and shows state-of-the-art test
accuracy on the Stanford Sentiment Treebank (SST), Multi-Genre natural language
inference (MultiNLI), Sentences Involving Compositional Knowledge (SICK),
Customer Review, MPQA, TREC question-type classification and Subjectivity
(SUBJ) datasets.

提出了一种新颖的方向性多维度自注意力机制 DiSAN，用于句子编码，相对于复杂的 RNN 模型，它在预测质量和时间效率方面表现更好，并在多个数据集上得到了最优的测试准确性表现。