In this paper, we study the problem of watermarking large language models
(LLMs). We consider the trade-off between model distortion and detection
ability and formulate it as a constrained optimization problem based on the
green-red algorithm of Kirchenbauer et al. (2023a). We show that the optimal
solution to the optimization problem enjoys a nice analytical property which
provides a better understanding and inspires the algorithm design for the
watermarking process. We develop an online dual gradient ascent watermarking
algorithm in light of this optimization formulation and prove its asymptotic
Pareto optimality between model distortion and detection ability. Such a result
guarantees an averaged increased green list probability and henceforth
detection ability explicitly (in contrast to previous results). Moreover, we
provide a systematic discussion on the choice of the model distortion metrics
for the watermarking problem. We justify our choice of KL divergence and
present issues with the existing criteria of ``distortion-free'' and
perplexity. Finally, we empirically evaluate our algorithms on extensive
datasets against benchmark algorithms.

本文研究了大型语言模型（LLMs）的水印问题，并将其模型畸变和检测能力之间的权衡视为一个基于 Kirchenbauer 等人（2023a）的绿 - 红算法的约束优化问题。通过该优化问题的最优解，我们证明了其具有良好的解析特性，从而更好地理解并启发了水印过程的算法设计。在此优化公式的基础上，我们开发了一种在线对偶梯度上升水印算法，并证明了其在模型畸变和检测能力之间的渐近帕累托最优性。这样的结果保证了平均增加的绿色列表概率和因此明确的检测能力（与之前的结果相比）。此外，我们对水印问题中模型畸变度量的选择进行了系统讨论。我们证明了选择 KL 散度的合理性，并介绍了现有的 “无畸变” 和困惑度标准存在的问题。最后，我们通过对广泛数据集的对比算法进行了实证评估。