The recent dominance of distributional language representation models has
elicited a variety of questions regarding their capabilities and intrinsic
properties, one of which is the manifestation of compositional phenomena in
natural language, which has significant implications towards explainability and
safety/fairness in the use of such models. While most current research on
compositionality has been directed towards improving performance of the
representations on similarity tasks, this work proposes a methodology for
measuring the presence of compositional behaviour in contemporary language
models related to adjectival modifier phenomena in adjective-noun phrases. Our
results show that current neural language models do not behave consistently
according to the linguistic theories with regard to the evaluated intersective
property, but on the other hand, the differences between adjective categories
are noticeable in single adjective interactions, indicating that such
differences are encoded in individual word representations, but they do not
transfer generally in the expected way to the compositions. This raises the
question of whether current language models are not capable of capturing the
true underlying distributional properties of language, or whether linguistic
theories from Montagovian tradition do not hold to distributional scrutiny.

本研究提出一种新的方法来衡量当今语言模型中形容词修饰现象的组合表现，研究结果表明当代神经语言模型在考虑下文影响时存在显著的差异，并且词类之间存在差异但在组合中无法表现，这引起了当前语言模型是否能够捕捉到真正的语言分布特性的质疑。

神经语言模型中的蒙塔古语义学和修饰一致性测量

Montague semantics and modifier consistency measurement in neural language models

Sentiment analysis is directly affected by compositional phenomena in
language that act on the prior polarity of the words and phrases found in the
text. Negation is the most prevalent of these phenomena and in order to
correctly predict sentiment, a classifier must be able to identify negation and
disentangle the effect that its scope has on the final polarity of a text. This
paper proposes a multi-task approach to explicitly incorporate information
about negation in sentiment analysis, which we show outperforms learning
negation implicitly in a data-driven manner. We describe our approach, a
cascading neural architecture with selective sharing of LSTM layers, and show
that explicitly training the model with negation as an auxiliary task helps
improve the main task of sentiment analysis. The effect is demonstrated across
several different standard English-language data sets for both tasks and we
analyze several aspects of our system related to its performance, varying types
and amounts of input data and different multi-task setups.

本文提出了一种多任务方法，通过将否定信息显式地纳入情感分析模型中，以优于隐式学习否定。该方法采用级联神经体系结构，并选择性共享 LSTM 层，同时在多个标准英语数据集上分析了该方法的性能，包括其多种不同设置下的输入数据类型和数量。