State-of-the-art language generation models can degenerate when applied to
open-ended generation problems such as text completion, story generation, or
dialog modeling. This degeneration usually shows up in the form of incoherence,
lack of vocabulary diversity, and self-repetition or copying from the context.
In this paper, we postulate that ``human-like'' generations usually lie in a
narrow and nearly flat entropy band, and violation of these entropy bounds
correlates with degenerate behavior. Our experiments show that this stable
narrow entropy zone exists across models, tasks, and domains and confirm the
hypothesis that violations of this zone correlate with degeneration. We then
use this insight to propose an entropy-aware decoding algorithm that respects
these entropy bounds resulting in less degenerate, more contextual, and
"human-like" language generation in open-ended text generation settings.

本论文提出，当语言生成模型用于文本完成，故事生成或对话建模等开放性文本生成问题时，最先进的语言生成模型可能会退化。我们发现近似平坦熵带内的生成更为 “人类化”，而模型违反熵的这些限制通常会导致退化。因此，我们提出了一种基于熵的解码算法，以生成更具语境感和 “人类化” 的文本。

稳定熵假设和熵感知解码：鲁棒自然语言生成的分析与算法

The Stable Entropy Hypothesis and Entropy-Aware Decoding: An Analysis and Algorithm for Robust Natural Language Generation

Humans often employ figurative language use in communication, including
during interactions with dialog systems. Thus, it is important for real-world
dialog systems to be able to handle popular figurative language constructs like
metaphor and simile. In this work, we analyze the performance of existing
dialog models in situations where the input dialog context exhibits use of
figurative language. We observe large gaps in handling of figurative language
when evaluating the models on two open domain dialog datasets. When faced with
dialog contexts consisting of figurative language, some models show very large
drops in performance compared to contexts without figurative language. We
encourage future research in dialog modeling to separately analyze and report
results on figurative language in order to better test model capabilities
relevant to real-world use. Finally, we propose lightweight solutions to help
existing models become more robust to figurative language by simply using an
external resource to translate figurative language to literal (non-figurative)
forms while preserving the meaning to the best extent possible.

研究发现现有的对话模型在处理比喻和比拟等修辞语言时存在很大的不足，提出利用外部资源将比喻语言转化为字面意思以提高模型鲁棒性的解决方案。

探究对常见比喻语言结构的对话模型的鲁棒性

Investigating Robustness of Dialog Models to Popular Figurative Language  Constructs

Due to the significance and value in human-computer interaction and natural
language processing, task-oriented dialog systems are attracting more and more
attention in both academic and industrial communities. In this paper, we survey
recent advances and challenges in task-oriented dialog systems. We also discuss
three critical topics for task-oriented dialog systems: (1) improving data
efficiency to facilitate dialog modeling in low-resource settings, (2) modeling
multi-turn dynamics for dialog policy learning to achieve better
task-completion performance, and (3) integrating domain ontology knowledge into
the dialog model. Besides, we review the recent progresses in dialog evaluation
and some widely-used corpora. We believe that this survey, though incomplete,
can shed a light on future research in task-oriented dialog systems.

本文探讨了任务导向对话系统的最新研究进展和挑战，特别在三个关键领域进行了深入讨论：提高数据效率、模拟多回合动态以优化任务完成性能，以及将领域本体知识与对话模型集成。此外，还评估了最近的对话进展和一些常用的语料库。我们相信，这虽然不是一个完整的调查，但它仍然可以为未来任务导向对话系统的研究提供启示。