Independence assumptions during sequence generation can speed up inference,
but parallel generation of highly inter-dependent tokens comes at a cost in
quality. Instead of assuming independence between neighbouring tokens
(semi-autoregressive decoding, SA), we take inspiration from bidirectional
sequence generation and introduce a decoder that generates target words from
the left-to-right and right-to-left directions simultaneously. We show that we
can easily convert a standard architecture for unidirectional decoding into a
bidirectional decoder by simply interleaving the two directions and adapting
the word positions and self-attention masks. Our interleaved bidirectional
decoder (IBDecoder) retains the model simplicity and training efficiency of the
standard Transformer, and on five machine translation tasks and two document
summarization tasks, achieves a decoding speedup of ~2X compared to
autoregressive decoding with comparable quality. Notably, it outperforms
left-to-right SA because the independence assumptions in IBDecoder are more
felicitous. To achieve even higher speedups, we explore hybrid models where we
either simultaneously predict multiple neighbouring tokens per direction, or
perform multi-directional decoding by partitioning the target sequence. These
methods achieve speedups to 4X-11X across different tasks at the cost of <1
BLEU or <0.5 ROUGE (on average). Source code is released at
this https URL

本文提出了一种交叉双向的解码器，它能同时进行从左到右和从右到左的目标词语生成，并且相较于自回归解码等传统解码方式在保持质量的情况下能实现达到 2 倍的解码速度提升。此外，作者还探索了同时预测多个相邻 token 的方法，以及根据目标序列进行分区解码的多向模型，这些方法也能够在不损失太多准确度的情况下进一步提高解码速度。

快速交替双向序列生成

Fast Interleaved Bidirectional Sequence Generation

Standard decoders for neural machine translation autoregressively generate a
single target token per time step, which slows inference especially for long
outputs. While architectural advances such as the Transformer fully parallelize
the decoder computations at training time, inference still proceeds
sequentially. Recent developments in non- and semi- autoregressive decoding
produce multiple tokens per time step independently of the others, which
improves inference speed but deteriorates translation quality. In this work, we
propose the syntactically supervised Transformer (SynST), which first
autoregressively predicts a chunked parse tree before generating all of the
target tokens in one shot conditioned on the predicted parse. A series of
controlled experiments demonstrates that SynST decodes sentences ~ 5x faster
than the baseline autoregressive Transformer while achieving higher BLEU scores
than most competing methods on En-De and En-Fr datasets.

本文介绍了一种基于句法指导的 Transformer 模型（SynST），它可以在预测出分块解析树之后自动并行地生成目标语言的所有标记，从而提高了翻译速度。一系列的实验表明，相比基线的自回归 Transformer 模型，SynST 模型在 En-De 和 En-Fr 数据集上实现了更高的 BLEU 分数，且速度快了约 5 倍。