This paper presents the results of the RepEval 2017 Shared Task, which
evaluated neural network sentence representation learning models on the
Multi-Genre Natural Language Inference corpus (MultiNLI) recently introduced by
Williams et al. (2017). All of the five participating teams beat the
bidirectional LSTM (BiLSTM) and continuous bag of words baselines reported in
Williams et al.. The best single model used stacked BiLSTMs with residual
connections to extract sentence features and reached 74.5% accuracy on the
genre-matched test set. Surprisingly, the results of the competition were
fairly consistent across the genre-matched and genre-mismatched test sets, and
across subsets of the test data representing a variety of linguistic phenomena,
suggesting that all of the submitted systems learned reasonably
domain-independent representations for sentence meaning.

本文介绍了 RepEval 2017 共享任务的结果，该任务评估了神经网络句子表示学习模型在 MultiNLI 语料库上的表现。结果表明，所有五个参赛团队都击败了以前的标记方法。最佳单一模型使用堆叠 BiLSTM 和残差连接来提取句子特征，并在与体裁匹配的测试集上达到了 74.5% 的准确率。结果表明，提交的所有系统对于句子意义的领域独立表示都有着合理的学习能力。

RepEval 2017 共享任务：基于句子表示的多体裁自然语言推断

The RepEval 2017 Shared Task: Multi-Genre Natural Language Inference  with Sentence Representations

In this paper we present the model used by the team Rivercorners for the 2017
RepEval shared task. First, our model separately encodes a pair of sentences
into variable-length representations by using a bidirectional LSTM. Later, it
creates fixed-length raw representations by means of simple aggregation
functions, which are then refined using an attention mechanism. Finally it
combines the refined representations of both sentences into a single vector to
be used for classification. With this model we obtained test accuracies of
72.057% and 72.055% in the matched and mismatched evaluation tracks
respectively, outperforming the LSTM baseline, and obtaining performances
similar to a model that relies on shared information between sentences (ESIM).
When using an ensemble both accuracies increased to 72.247% and 72.827%
respectively.

用 bidirectional LSTM 将一对句子分别编码成变长表示，再通过简单聚合函数创建固定长度的未经处理的表示。接着使用注意机制对这些表示进行细化处理。最后，将两个句子的细化表示相结合成为一个用于分类的向量，并且使用集成方法来提高模型的准确性，最终比 LSTM 基线表现更好，类似于 ESIM 模型的表现。

通过注意力机制提炼原始句子表示，用于文本蕴含识别

Refining Raw Sentence Representations for Textual Entailment Recognition  via Attention

There is a lot of research interest in encoding variable length sentences
into fixed length vectors, in a way that preserves the sentence meanings. Two
common methods include representations based on averaging word vectors, and
representations based on the hidden states of recurrent neural networks such as
LSTMs. The sentence vectors are used as features for subsequent machine
learning tasks or for pre-training in the context of deep learning. However,
not much is known about the properties that are encoded in these sentence
representations and about the language information they capture. We propose a
framework that facilitates better understanding of the encoded representations.
We define prediction tasks around isolated aspects of sentence structure
(namely sentence length, word content, and word order), and score
representations by the ability to train a classifier to solve each prediction
task when using the representation as input. We demonstrate the potential
contribution of the approach by analyzing different sentence representation
mechanisms. The analysis sheds light on the relative strengths of different
sentence embedding methods with respect to these low level prediction tasks,
and on the effect of the encoded vector's dimensionality on the resulting
representations.

通过对句子结构的预测任务来评估句子嵌入向量的表现，探讨了句子嵌入方法的相对优劣和向量维度对嵌入效果的影响。