In the encoder-decoder architecture for neural machine translation (NMT), the
hidden states of the recurrent structures in the encoder and decoder carry the
crucial information about the sentence.These vectors are generated by
parameters which are updated by back-propagation of translation errors through
time. We argue that propagating errors through the end-to-end recurrent
structures are not a direct way of control the hidden vectors. In this paper,
we propose to use word predictions as a mechanism for direct supervision. More
specifically, we require these vectors to be able to predict the vocabulary in
target sentence. Our simple mechanism ensures better representations in the
encoder and decoder without using any extra data or annotation. It is also
helpful in reducing the target side vocabulary and improving the decoding
efficiency. Experiments on Chinese-English and German-English machine
translation tasks show BLEU improvements by 4.53 and 1.3, respectively

本文提出了一种通过预测目标语句中词汇的方法，直接对编码器和解码器中隐藏状态向量进行监督，从而提高了表示质量和机器翻译的效率，实验证明在中英翻译任务和德英翻译任务中 BLEU 得分分别提高了 4.53 和 1.3。

基于单词预测的神经机器翻译

Neural Machine Translation with Word Predictions

We propose a finite-state transducer (FST) representation for the models used
to decode keyboard inputs on mobile devices. Drawing from learnings from the
field of speech recognition, we describe a decoding framework that can satisfy
the strict memory and latency constraints of keyboard input. We extend this
framework to support functionalities typically not present in speech
recognition, such as literal decoding, autocorrections, word completions, and
next word predictions.
We describe the general framework of what we call for short the keyboard "FST
decoder" as well as the implementation details that are new compared to a
speech FST decoder. We demonstrate that the FST decoder enables new UX features
such as post-corrections. Finally, we sketch how this decoder can support
advanced features such as personalization and contextualization.

本文提出一种基于有限状态转换器的键盘输入解码模型，通过借鉴语音识别的实践经验，旨在满足移动设备键盘输入的严格内存及延迟要求，并且支持文字解码、自动纠正、单词补全和下一个词预测等本文未涉及的功能，并演示了新的 UX 特性，例如后置纠正，并概述了该解码器如何支持个性化和情境化的高级功能。