A recent line of work in NLP focuses on the (dis)ability of models to generalise compositionally for artificial languages. However, when considering natural language tasks, the data involved is not strictly, or locally, compositional. Quantifying the compositionality of data is a challenging task, which has been investigated primarily for short utterances. We use recursive neural models (Tree-LSTMs) with bottlenecks that limit the transfer of information between nodes. We illustrate that comparing data's representations in models with and without the bottleneck can be used to produce a compositionality metric. The procedure is applied to the evaluation of arithmetic expressions using synthetic data, and sentiment classification using natural language data. We demonstrate that compression through a bottleneck impacts non-compositional examples disproportionately and then use the bottleneck compositionality metric (BCM) to distinguish compositional from non-compositional samples, yielding a compositionality ranking over a dataset.

本研究使用递归神经网络模型和限制信息传递的瓶颈机制，通过比较有无瓶颈情况下模型对数据的表示，提出了一种衡量复合性的度量标准，并基于此标准实现了一个数据集的复合性排序，应用于算术表达式合成数据和情感分类自然语言数据。

具有瓶颈的递归神经网络诊断（非）合成性