Previous works show that deep NLP models are not always conceptually sound:
they do not always learn the correct linguistic concepts. Specifically, they
can be insensitive to word order. In order to systematically evaluate models
for their conceptual soundness with respect to word order, we introduce a new
explanation method for sequential data: Order-sensitive Shapley Values (OSV).
We conduct an extensive empirical evaluation to validate the method and surface
how well various deep NLP models learn word order. Using synthetic data, we
first show that OSV is more faithful in explaining model behavior than
gradient-based methods. Second, applying to the HANS dataset, we discover that
the BERT-based NLI model uses only the word occurrences without word orders.
Although simple data augmentation improves accuracy on HANS, OSV shows that the
augmented model does not fundamentally improve the model's learning of order.
Third, we discover that not all sentiment analysis models learn negation
properly: some fail to capture the correct syntax of the negation construct.
Finally, we show that pretrained language models such as BERT may rely on the
absolute positions of subject words to learn long-range Subject-Verb Agreement.
With each NLP task, we also demonstrate how OSV can be leveraged to generate
adversarial examples.

研究探讨使用新的解释方法 -- 顺序敏感的 Shapley Value 方法评估深度自然语言处理模型对单词顺序的概念正确性的影响，并发现在处理 HANS 数据集和情感分析任务时，某些深度模型对否定构造和主语动词一致性的语法和顺序理解存在局限性。