Structured prediction energy networks (SPENs; Belanger & McCallum 2016) use
neural network architectures to define energy functions that can capture
arbitrary dependencies among parts of structured outputs. Prior work used
gradient descent for inference, relaxing the structured output to a set of
continuous variables and then optimizing the energy with respect to them. We
replace this use of gradient descent with a neural network trained to
approximate structured argmax inference. This "inference network" outputs
continuous values that we treat as the output structure. We develop
large-margin training criteria for joint training of the structured energy
function and inference network. On multi-label classification we report
speed-ups of 10-60x compared to (Belanger et al, 2017) while also improving
accuracy. For sequence labeling with simple structured energies, our approach
performs comparably to exact inference while being much faster at test time. We
then demonstrate improved accuracy by augmenting the energy with a "label
language model" that scores entire output label sequences, showing it can
improve handling of long-distance dependencies in part-of-speech tagging.
Finally, we show how inference networks can replace dynamic programming for
test-time inference in conditional random fields, suggestive for their general
use for fast inference in structured settings.

本文提出了一种基于神经网络的结构化预测能量网络方法，使用经过训练的神经网络来进行结构化 argmax 推理，同时开发了大边界训练准则以实现对能量函数和推理网络的联合训练。在多标签分类和序列标注等任务中，该方法能够在保持或提高准确性的同时大幅提升推理速度。