We describe a neural transducer that maintains the flexibility of standard
sequence-to-sequence (seq2seq) models while incorporating hierarchical phrases
as a source of inductive bias during training and as explicit constraints
during inference. Our approach trains two models: a discriminative parser based
on a bracketing transduction grammar whose derivation tree hierarchically
aligns source and target phrases, and a neural seq2seq model that learns to
translate the aligned phrases one-by-one. We use the same seq2seq model to
translate at all phrase scales, which results in two inference modes: one mode
in which the parser is discarded and only the seq2seq component is used at the
sequence-level, and another in which the parser is combined with the seq2seq
model. Decoding in the latter mode is done with the cube-pruned CKY algorithm,
which is more involved but can make use of new translation rules during
inference. We formalize our model as a source-conditioned synchronous grammar
and develop an efficient variational inference algorithm for training. When
applied on top of both randomly initialized and pretrained seq2seq models, we
find that both inference modes performs well compared to baselines on small
scale machine translation benchmarks.

本研究提出一种基于神经网络的译码器，结合层次短语作为训练的归纳偏差和显式约束，在维持标准序列到序列 (seq2seq) 模型的灵活性的同时。通过训练一个基于括号转录文法的判别式解析器，以层次化地对齐源和目标语短语，并使用一个神经 seq2seq 模型逐个地翻译这些对齐短语。该模型提出两种推理模式：一种只依赖于 seq2seq 模型进行序列级别的翻译，另一种则结合了解析器和 seq2seq 模型。研究结果表明，在小规模机器翻译基准测试中，该方法与对照方法相比表现良好。