Constituency Parse Extraction from Pre-trained Language Models (CPE-PLM) is a
recent paradigm that attempts to induce constituency parse trees relying only
on the internal knowledge of pre-trained language models. While attractive in
the perspective that similar to in-context learning, it does not require
task-specific fine-tuning, the practical effectiveness of such an approach
still remains unclear, except that it can function as a probe for investigating
language models' inner workings. In this work, we mathematically reformulate
CPE-PLM and propose two advanced ensemble methods tailored for it,
demonstrating that the new parsing paradigm can be competitive with common
unsupervised parsers by introducing a set of heterogeneous PLMs combined using
our techniques. Furthermore, we explore some scenarios where the trees
generated by CPE-PLM are practically useful. Specifically, we show that CPE-PLM
is more effective than typical supervised parsers in few-shot settings.

本文提出了两种预先训练的语言模型的高级集成方法，证明这种新的解析范例可以通过引入一组异构的 PLM 使用我们的技术与常规的无监督解析器竞争。 此外，本文还探讨了 CPE-PLM 生成的树在实际中有用的一些情况，特别是在少量数据学习环境中，CPE-PLM 比典型的监督解析器更有效。

重新审视预训练语言模型提取短语结构句法分析的实际有效性

Revisiting the Practical Effectiveness of Constituency Parse Extraction from Pre-trained Language Models

Recently, there has been an increasing interest in unsupervised parsers that
optimize semantically oriented objectives, typically using reinforcement
learning. Unfortunately, the learned trees often do not match actual syntax
trees well. Shen et al. (2018) propose a structured attention mechanism for
language modeling (PRPN), which induces better syntactic structures but relies
on ad hoc heuristics. Also, their model lacks interpretability as it is not
grounded in parsing actions. In our work, we propose an imitation learning
approach to unsupervised parsing, where we transfer the syntactic knowledge
induced by the PRPN to a Tree-LSTM model with discrete parsing actions. Its
policy is then refined by Gumbel-Softmax training towards a semantically
oriented objective. We evaluate our approach on the All Natural Language
Inference dataset and show that it achieves a new state of the art in terms of
parsing $F$-score, outperforming our base models, including the PRPN.

我们提出了一种模仿学习的无监督句法分析方法，将 PRPN 诱导的句法知识转移到带有离散句法分析操作的 Tree-LSTM 模型上，并通过 Gumbel-Softmax 训练不断优化其策略以实现更好的语义目标。在 All-NLI 数据集上进行实验，证明我们的方法在句法得分上超过了基准模型，包括 PRPN。