Exposure bias poses a common challenge in numerous natural language
processing tasks, particularly in the dialog generation. In response to this
issue, researchers have devised various techniques, among which scheduled
sampling has proven to be an effective method for mitigating exposure bias.
However, the existing state-of-the-art scheduled sampling methods solely
consider the current sampling words' quality for threshold truncation sampling,
which overlooks the importance of sentence-level information and the method of
threshold truncation warrants further discussion. In this paper, we propose a
bilevel scheduled sampling model that takes the sentence-level information into
account and incorporates it with word-level quality. To enhance sampling
diversity and improve the model's adaptability, we propose a smooth function
that maps the combined result of sentence-level and word-level information to
an appropriate range, and employ probabilistic sampling based on the mapped
values instead of threshold truncation. Experiments conducted on the
DailyDialog and PersonaChat datasets demonstrate the effectiveness of our
proposed methods, which significantly alleviate the exposure bias problem and
outperform state-of-the-art scheduled sampling methods.

本文提出了一种双层预训练采样模型，结合句子级信息和单词级质量，并利用平滑函数将两者的综合结果映射到适当的范围，基于映射值进行概率采样。在 DailyDialog 和 PersonaChat 数据集上的实验证明了我们提出的方法的有效性，明显缓解了暴露偏差问题，并超过了最先进的预训练采样方法。