Attention-based sequence-to-sequence model has proved successful in Neural
Machine Translation (NMT). However, the attention without consideration of
decoding history, which includes the past information in the decoder and the
attention mechanism, often causes much repetition. To address this problem, we
propose the decoding-history-based Adaptive Control of Attention (ACA) for the
NMT model. ACA learns to control the attention by keeping track of the decoding
history and the current information with a memory vector, so that the model can
take the translated contents and the current information into consideration.
Experiments on Chinese-English translation and the English-Vietnamese
translation have demonstrated that our model significantly outperforms the
strong baselines. The analysis shows that our model is capable of generating
translation with less repetition and higher accuracy. The code will be
available at this https URL

提出了一种基于解码历史的自适应注意力控制机制，可以解决注意力在缺乏解码历史信息的情况下导致频繁重复的问题，在中英文和英越翻译任务上均表现出较高的翻译准确性，可生成较少重复的翻译。

基于解码历史的自适应控制注意力的神经机器翻译

Decoding-History-Based Adaptive Control of Attention for Neural Machine  Translation

Learning and generating Chinese poems is a charming yet challenging task.
Traditional approaches involve various language modeling and machine
translation techniques, however, they perform not as well when generating poems
with complex pattern constraints, for example Song iambics, a famous type of
poems that involve variable-length sentences and strict rhythmic patterns. This
paper applies the attention-based sequence-to-sequence model to generate
Chinese Song iambics. Specifically, we encode the cue sentences by a
bi-directional Long-Short Term Memory (LSTM) model and then predict the entire
iambic with the information provided by the encoder, in the form of an
attention-based LSTM that can regularize the generation process by the fine
structure of the input cues. Several techniques are investigated to improve the
model, including global context integration, hybrid style training, character
vector initialization and adaptation. Both the automatic and subjective
evaluation results show that our model indeed can learn the complex structural
and rhythmic patterns of Song iambics, and the generation is rather successful.

本文利用基于注意力机制的序列到序列模型生成中国宋词。通过双向 LSTM 模型对提示句进行编码，再通过基于注意力的 LSTM 模型预测整个倚仗，可以通过输入线索的细微结构来规范化生成过程。研究了几种技术来改进模型，包括全局上下文整合，混合式风格训练，字符向量初始化和适应性。通过自动和主观评估结果表明，我们的模型确实可以学习到宋词复杂的结构和韵律模式，且生成成功。