Dynamic oracles provide strong supervision for training constituency parsers
with exploration, but must be custom defined for a given parser's transition
system. We explore using a policy gradient method as a parser-agnostic
alternative. In addition to directly optimizing for a tree-level metric such as
F1, policy gradient has the potential to reduce exposure bias by allowing
exploration during training; moreover, it does not require a dynamic oracle for
supervision. On four constituency parsers in three languages, the method
substantially outperforms static oracle likelihood training in almost all
settings. For parsers where a dynamic oracle is available (including a novel
oracle which we define for the transition system of Dyer et al. 2016), policy
gradient typically recaptures a substantial fraction of the performance gain
afforded by the dynamic oracle.

本研究探讨使用策略梯度方法作为独立于特定解析器转移系统的替代动态预言机的多元组监督方法，以直接优化 F1 指标并在训练期间进行探索，从而减少暴露偏差。在三种不同语言的四个成分解析器中，该方法在几乎所有情况下都比静态预言机似然训练方法表现出更高的效果。