Autoregressive decoding limits the efficiency of transformers for Machine
Translation (MT). The community proposed specific network architectures and
learning-based methods to solve this issue, which are expensive and require
changes to the MT model, trading inference speed at the cost of the translation
quality. In this paper, we propose to address the problem from the point of
view of decoding algorithms, as a less explored but rather compelling
direction. We propose to reframe the standard greedy autoregressive decoding of
MT with a parallel formulation leveraging Jacobi and Gauss-Seidel fixed-point
iteration methods for fast inference. This formulation allows to speed up
existing models without training or modifications while retaining translation
quality. We present three parallel decoding algorithms and test them on
different languages and models showing how the parallelization introduces a
speedup up to 38% w.r.t. the standard autoregressive decoding and nearly 2x
when scaling the method on parallel resources. Finally, we introduce a decoding
dependency graph visualizer (DDGviz) that let us see how the model has learned
the conditional dependence between tokens and inspect the decoding procedure.

本文提出使用雅可比和高斯 - 塞德尔迭代方法对机器翻译的标准贪心自回归解码进行并行重构，以解决 transformers 对机器翻译的效率问题，实现了速度提高，而翻译质量保持。三种并行解码算法被提出并在不同语言和模型上进行了测试，速度提升最多可达 38％，在并行资源上实现的速度几乎达到 2 倍。同时，为了更好地理解模型的条件依赖关系和检查解码过程，提出了解码依赖图可视化器（DDGviz）

通过并行解码加速 Transformer 推理以进行翻译

Accelerating Transformer Inference for Translation via Parallel Decoding

Feedforward computation, such as evaluating a neural network or sampling from
an autoregressive model, is ubiquitous in machine learning. The sequential
nature of feedforward computation, however, requires a strict order of
execution and cannot be easily accelerated with parallel computing. To enable
parallelization, we frame the task of feedforward computation as solving a
system of nonlinear equations. We then propose to find the solution using a
Jacobi or Gauss-Seidel fixed-point iteration method, as well as hybrid methods
of both. Crucially, Jacobi updates operate independently on each equation and
can be executed in parallel. Our method is guaranteed to give exactly the same
values as the original feedforward computation with a reduced (or equal) number
of parallelizable iterations, and hence reduced time given sufficient parallel
computing power. Experimentally, we demonstrate the effectiveness of our
approach in accelerating (i) backpropagation of RNNs, (ii) evaluation of
DenseNets, and (iii) autoregressive sampling of MADE and PixelCNN++, with
speedup factors between 2.1 and 26 under various settings.

本文章提出了使用 Jacobi 或 Gauss-Seidel 定点迭代方法来并行化前馈计算任务，以实现在神经网络评估和自回归模型采样等机器学习任务中的加速。实验表明，该方法在加速反向传播和评估 DenseNets 和 autoregressive sampling 任务中具有高效性，并在不同设置下显示出 2.1 至 26 倍的加速因子。