We introduce a frustratingly simple, super efficient and surprisingly
effective decoding method, which we call Frustratingly Simple Decoding (FSD),
for neural text generation. The idea behind FSD is straightforward: we build an
anti-LM based on previously generated text and use this anti-LM to penalize
future generation of what has been generated. The anti-LM can be implemented as
simple as an n-gram language model or a vectorized variant. In this way, FSD
introduces no extra model parameters and negligible computational overhead (FSD
can be as fast as greedy search). Despite the simplicity, FSD is surprisingly
effective; Experiments show that FSD can outperform the canonical methods to
date (i.e., nucleus sampling) as well as several strong baselines that were
proposed recently.

本文介绍了一种名为 Frustratingly Simple Decoding (FSD) 的超级高效的解码方法，通过构建一个基于先前生成文本的 anti-LM 来惩罚未来生成的文本，实现了神经文本生成的更好效果，且不增加额外的模型参数和计算开销。

神经文本生成的一种令人沮丧的简化解码方法

A Frustratingly Simple Decoding Method for Neural Text Generation

Recent work on word ordering has argued that syntactic structure is
important, or even required, for effectively recovering the order of a
sentence. We find that, in fact, an n-gram language model with a simple
heuristic gives strong results on this task. Furthermore, we show that a long
short-term memory (LSTM) language model is even more effective at recovering
order, with our basic model outperforming a state-of-the-art syntactic model by
11.5 BLEU points. Additional data and larger beams yield further gains, at the
expense of training and search time.

研究表明，使用 n-gram 语言模型和 LSTM 语言模型可以更有效地恢复句子的顺序，比一些最先进的句法模型表现更好。

无需语法的词序排列

Word Ordering Without Syntax

Lexical substitutes have found use in areas such as paraphrasing, text
simplification, machine translation, word sense disambiguation, and part of
speech induction. However the computational complexity of accurately
identifying the most likely substitutes for a word has made large scale
experiments difficult. In this paper I introduce a new search algorithm,
FASTSUBS, that is guaranteed to find the K most likely lexical substitutes for
a given word in a sentence based on an n-gram language model. The computation
is sub-linear in both K and the vocabulary size V. An implementation of the
algorithm and a dataset with the top 100 substitutes of each token in the WSJ
section of the Penn Treebank are available at this http URL

本论文介绍了一种基于 n-gram 语言模型的新搜索算法 FASTSUBS, 用于精确定位给定句子中最有可能的词汇替换，可应用于诸如文本简化、机器翻译和词性归纳等领域，算法计算复杂度在 K 和词汇表大小 V 方面具有亚线性的特点。