A series of recent papers has used a parsing algorithm due to Shen et al.
(2018) to recover phrase-structure trees based on proxies for "syntactic
depth." These proxy depths are obtained from the representations learned by
recurrent language models augmented with mechanisms that encourage the
(unsupervised) discovery of hierarchical structure latent in natural language
sentences. Using the same parser, we show that proxies derived from a
conventional LSTM language model produce trees comparably well to the
specialized architectures used in previous work. However, we also provide a
detailed analysis of the parsing algorithm, showing (1) that it is
incomplete---that is, it can recover only a fraction of possible trees---and
(2) that it has a marked bias for right-branching structures which results in
inflated performance in right-branching languages like English. Our analysis
shows that evaluating with biased parsing algorithms can inflate the apparent
structural competence of language models.

本文使用 Shen 等人 (2018) 的句法分析算法来恢复基于 “句法深度” 代理的短语结构树，并使用常规 LSTM 语言模型提供的代理进行比较，发现偏向于右分支结构的解析算法可以夸大语言模型的表现能力。