Latent structure models are a powerful tool for modeling language data: they can mitigate the error propagation and annotation bottleneck in pipeline systems, while simultaneously uncovering linguistic insights about the data. One challenge with end-to-end training of these models is the argmax operation, which has null gradient. In this paper, we focus on surrogate gradients, a popular strategy to deal with this problem. We explore latent structure learning through the angle of pulling back the downstream learning objective. In this paradigm, we discover a principled motivation for both the straight-through estimator (STE) as well as the recently-proposed SPIGOT - a variant of STE for structured models. Our perspective leads to new algorithms in the same family. We empirically compare the known and the novel pulled-back estimators against the popular alternatives, yielding new insight for practitioners and revealing intriguing failure cases.

本文讨论了拉回下游学习目标方法来探索潜在结构学习的原理，从而发现了STE和SPIGOT的原则动机，这导致了相同家族中的新算法，并将已知的和新的拉回估计器与流行的选择进行了实证比较，为实践者提供了新的见识，并揭示了有趣的失败案例。

理解 SPIGOT 的机制：用作潜在结构学习的替代导数