Recent language models, especially those based on recurrent neural networks
(RNNs), make it possible to generate natural language from a learned
probability. Language generation has wide applications including machine
translation, summarization, question answering, conversation systems, etc.
Existing methods typically learn a joint probability of words conditioned on
additional information, which is (either statically or dynamically) fed to
RNN's hidden layer. In many applications, we are likely to impose hard
constraints on the generated texts, i.e., a particular word must appear in the
sentence. Unfortunately, existing approaches could not solve this problem. In
this paper, we propose a novel backward and forward language model. Provided a
specific word, we use RNNs to generate previous words and future words, either
simultaneously or asynchronously, resulting in two model variants. In this way,
the given word could appear at any position in the sentence. Experimental
results show that the generated texts are comparable to sequential LMs in
quality.

本研究提出了一种新的前向后向语言模型，能够在特定词的约束下生成自然语言文本，表现优于传统的顺序语言模型。