A recent line of work in NLP focuses on the (dis)ability of models to
generalise compositionally for artificial languages. However, when considering
natural language tasks, the data involved is not strictly, or locally,
compositional. Quantifying the compositionality of data is a challenging task,
which has been investigated primarily for short utterances. We use recursive
neural models (Tree-LSTMs) with bottlenecks that limit the transfer of
information between nodes. We illustrate that comparing data's representations
in models with and without the bottleneck can be used to produce a
compositionality metric. The procedure is applied to the evaluation of
arithmetic expressions using synthetic data, and sentiment classification using
natural language data. We demonstrate that compression through a bottleneck
impacts non-compositional examples disproportionately and then use the
bottleneck compositionality metric (BCM) to distinguish compositional from
non-compositional samples, yielding a compositionality ranking over a dataset.

本研究使用递归神经网络模型和限制信息传递的瓶颈机制，通过比较有无瓶颈情况下模型对数据的表示，提出了一种衡量复合性的度量标准，并基于此标准实现了一个数据集的复合性排序，应用于算术表达式合成数据和情感分类自然语言数据。

具有瓶颈的递归神经网络诊断（非）合成性

Recursive Neural Networks with Bottlenecks Diagnose (Non-)Compositionality

Recursive neural models, which use syntactic parse trees to recursively
generate representations bottom-up, are a popular architecture. But there have
not been rigorous evaluations showing for exactly which tasks this syntax-based
method is appropriate. In this paper we benchmark {\bf recursive} neural models
against sequential {\bf recurrent} neural models (simple recurrent and LSTM
models), enforcing apples-to-apples comparison as much as possible. We
investigate 4 tasks: (1) sentiment classification at the sentence level and
phrase level; (2) matching questions to answer-phrases; (3) discourse parsing;
(4) semantic relation extraction (e.g., {\em component-whole} between nouns).
Our goal is to understand better when, and why, recursive models can
outperform simpler models. We find that recursive models help mainly on tasks
(like semantic relation extraction) that require associating headwords across a
long distance, particularly on very long sequences. We then introduce a method
for allowing recurrent models to achieve similar performance: breaking long
sentences into clause-like units at punctuation and processing them separately
before combining. Our results thus help understand the limitations of both
classes of models, and suggest directions for improving recurrent models.

本文比较了基于语法树递归生成表征的递归神经模型与基于序列生成表征的循环神经模型（包括简单循环和 LSTM 模型）在情感分类、问答匹配、话语解析和语义关系提取等 4 个任务上的效果，并提出一种将长句子分解为类从句单元再分别处理的方法以提高循环模型的性能。研究结果表明递归模型在头词之间相隔较远、序列较长的任务上可优于循环模型，同时也揭示了两类模型的局限性以及未来改进方向。