Existing approaches to neural machine translation (NMT) generate the target
language sequence token by token from left to right. However, this kind of
unidirectional decoding framework cannot make full use of the target-side
future contexts which can be produced in a right-to-left decoding direction,
and thus suffers from the issue of unbalanced outputs. In this paper, we
introduce a synchronous bidirectional neural machine translation (SB-NMT) that
predicts its outputs using left-to-right and right-to-left decoding
simultaneously and interactively, in order to leverage both of the history and
future information at the same time. Specifically, we first propose a new
algorithm that enables synchronous bidirectional decoding in a single model.
Then, we present an interactive decoding model in which left-to-right
(right-to-left) generation does not only depend on its previously generated
outputs, but also relies on future contexts predicted by right-to-left
(left-to-right) decoding. We extensively evaluate the proposed SB-NMT model on
large-scale NIST Chinese-English, WMT14 English-German, and WMT18
Russian-English translation tasks. Experimental results demonstrate that our
model achieves significant improvements over the strong Transformer model by
3.92, 1.49 and 1.04 BLEU points respectively, and obtains the state-of-the-art
performance on Chinese-English and English-German translation tasks.

本研究提出了同步双向神经机器翻译模型，通过同时进行从左到右和从右到左解码，以相互交互的方式预测输出，从而利用历史和未来信息，并在大规模数据上实验证明其优于强的 Transformer 模型，达到了中英文和英德文翻译任务的最新性能。

同步双向神经机器翻译

Synchronous Bidirectional Neural Machine Translation

In sequence to sequence generation tasks (e.g. machine translation and
abstractive summarization), inference is generally performed in a left-to-right
manner to produce the result token by token. The neural approaches, such as
LSTM and self-attention networks, are now able to make full use of all the
predicted history hypotheses from left side during inference, but cannot
meanwhile access any future (right side) information and usually generate
unbalanced outputs in which left parts are much more accurate than right ones.
In this work, we propose a synchronous bidirectional inference model to
generate outputs using both left-to-right and right-to-left decoding
simultaneously and interactively. First, we introduce a novel beam search
algorithm that facilitates synchronous bidirectional decoding. Then, we present
the core approach which enables left-to-right and right-to-left decoding to
interact with each other, so as to utilize both the history and future
predictions simultaneously during inference. We apply the proposed model to
both LSTM and self-attention networks. In addition, we propose two strategies
for parameter optimization. The extensive experiments on machine translation
and abstractive summarization demonstrate that our synchronous bidirectional
inference model can achieve remarkable improvements over the strong baselines.

提出了一种同步双向推理模型，基于先前预测的左侧历史假设和右侧未来预测，同时利用从左到右和从右到左的解码生成输出，通过机器翻译和摘要生成等多个任务的广泛实验表明，其表现很好。