The strong general capabilities of Large Language Models (LLMs) bring
potential ethical risks if they are unrestrictedly accessible to malicious
users. Token-level watermarking inserts watermarks in the generated texts by
altering the token probability distributions with a private random number
generator seeded by its prefix tokens. However, this watermarking algorithm
alters the logits during generation, which can lead to a downgraded text
quality if it chooses to promote tokens that are less relevant given the input.
In this work, we propose to improve the quality of texts generated by a
watermarked language model by Watermarking with Importance Scoring (WIS). At
each generation step, we estimate the importance of the token to generate, and
prevent it from being impacted by watermarking if it is important for the
semantic correctness of the output. We further propose three methods to predict
importance scoring, including a perturbation-based method and two model-based
methods. Empirical experiments show that our method can generate texts with
better quality with comparable level of detection rate.

使用重要性评分的水印语言模型提高生成的文本质量，并提出三种预测重要性评分的方法。

通过词重要性评分提高带水印的大型语言模型的生成质量

Improving the Generation Quality of Watermarked Large Language Models  via Word Importance Scoring

As AI-generated text increasingly resembles human-written content, the
ability to detect machine-generated text becomes crucial. To address this
challenge, we present GPTWatermark, a robust and high-quality solution designed
to ascertain whether a piece of text originates from a specific model. Our
approach extends existing watermarking strategies and employs a fixed group
design to enhance robustness against editing and paraphrasing attacks. We show
that our watermarked language model enjoys strong provable guarantees on
generation quality, correctness in detection, and security against evasion
attacks. Experimental results on various large language models (LLMs) and
diverse datasets demonstrate that our method achieves superior detection
accuracy and comparable generation quality in perplexity, thus promoting the
responsible use of LLMs.

为了解决检测人工智能生成文本的问题，本研究提出了一种鲁棒且高质量的方案，名为 GPTWatermark，通过水印的方式确定文本的来源，并在大规模语言模型中展示了其稳定性和高检测准确性。