Large language models (LLMs) have shown impressive capabilities in adapting
to various tasks when provided with task-specific instructions. However, LLMs
using standard decoding strategies often struggle with deviations from the
inputs. Intuitively, compliant LLM outputs should reflect the information
present in the input, which can be measured by point-wise mutual information
(PMI) scores. Therefore, we propose Diver, a novel approach that enhances LLM
Decoding through span-level PMI verification. During inference, Diver first
identifies divergence steps that may lead to multiple candidate spans.
Subsequently, it calculates the PMI scores by assessing the log-likelihood
gains of the input if the candidate spans are generated. Finally, the optimal
span is selected based on the PMI re-ranked output distributions. We evaluate
our method across various downstream tasks, and empirical results demonstrate
that Diver significantly outperforms existing decoding methods in both
performance and versatility.

使用点对点的互信息评分，我们提出了一种增强大型语言模型解码的新方法：Diver。实验证明，Diver 在性能和适用性方面明显优于现有的解码方法。