We consider the emerging problem of identifying the presence and use of
watermarking schemes in widely used, publicly hosted, closed source large
language models (LLMs). We introduce a suite of baseline algorithms for
identifying watermarks in LLMs that rely on analyzing distributions of output
tokens and logits generated by watermarked and unmarked LLMs. Notably,
watermarked LLMs tend to produce distributions that diverge qualitatively and
identifiably from standard models. Furthermore, we investigate the
identifiability of watermarks at varying strengths and consider the tradeoffs
of each of our identification mechanisms with respect to watermarking scenario.
Along the way, we formalize the specific problem of identifying watermarks in
LLMs, as well as LLM watermarks and watermark detection in general, providing a
framework and foundations for studying them.

通过分析输出 token 和 logit 的分布，提出了一套基线算法来识别广泛使用的水印方案在大型语言模型中的存在和使用，并考虑了各种情况下的鉴别度和识别机制的权衡。同时，正式阐明了在大型语言模型和水印检测方面的特定问题。

辨识带水印的大型语言模型的基准

Baselines for Identifying Watermarked Large Language Models

We propose a framework for analyzing and comparing distributions, allowing us
to design statistical tests to determine if two samples are drawn from
different distributions. Our test statistic is the largest difference in
expectations over functions in the unit ball of a reproducing kernel Hilbert
space (RKHS). We present two tests based on large deviation bounds for the test
statistic, while a third is based on the asymptotic distribution of this
statistic. The test statistic can be computed in quadratic time, although
efficient linear time approximations are available. Several classical metrics
on distributions are recovered when the function space used to compute the
difference in expectations is allowed to be more general (eg. a Banach space).
We apply our two-sample tests to a variety of problems, including attribute
matching for databases using the Hungarian marriage method, where they perform
strongly. Excellent performance is also obtained when comparing distributions
over graphs, for which these are the first such tests.

提出一种基于重现核希尔伯特空间和大偏差界限的统计检验框架，可用于分析和比较分布，特别对于数据库属性匹配具有很好的性能表现。