An advantage of seq2seq abstractive summarization models is that they
generate text in a free-form manner, but this flexibility makes it difficult to
interpret model behavior. In this work, we analyze summarization decoders in
both blackbox and whitebox ways by studying on the entropy, or uncertainty, of
the model's token-level predictions. For two strong pre-trained models, PEGASUS
and BART on two summarization datasets, we find a strong correlation between
low prediction entropy and where the model copies tokens rather than generating
novel text. The decoder's uncertainty also connects to factors like sentence
position and syntactic distance between adjacent pairs of tokens, giving a
sense of what factors make a context particularly selective for the model's
next output token. Finally, we study the relationship of decoder uncertainty
and attention behavior to understand how attention gives rise to these observed
effects in the model. We show that uncertainty is a useful perspective for
analyzing summarization and text generation models more broadly.

通过对两个已预训练模型 PEGASUS 和 BART 进行分析，本文探讨了在两个文本摘要数据集上，token-level 预测的不确定性与模型规律的关系，发现模型在复制标记而非生成新文本的情况下，预测熵值较小；同时，不确定性与句子位置、相邻标记之间的语法距离等因素有关，并阐述了注意力机制对模型效果的影响。