Abuse of large language models reveals high risks as large language models
are being deployed at an astonishing speed. It is important to protect the
model weights to avoid malicious usage that violates licenses of open-source
large language models. This paper proposes a novel watermarking strategy that
plants watermarks in the quantization process of large language models without
pre-defined triggers during inference. The watermark works when the model is
used in the fp32 mode and remains hidden when the model is quantized to int8,
in this way, the users can only inference the model without further supervised
fine-tuning of the model. We successfully plant the watermark into open-source
large language model weights including GPT-Neo and LLaMA. We hope our proposed
method can provide a potential direction for protecting model weights in the
era of large language model applications.

这篇论文提出了一种新颖的水印策略，通过在大型语言模型的量化过程中植入水印来保护模型权重，以避免恶意使用违反开源大型语言模型的许可证。该水印只在使用 fp32 模式的模型中起作用，并在将模型量化为 int8 时保持隐藏，使用户只能推理模型而无法进一步对模型进行监督微调。我们成功地将水印嵌入开源大型语言模型的权重中，包括 GPT-Neo 和 LLaMA。希望我们提出的方法能够为保护大型语言模型权重在大型语言模型应用时提供潜在方向。

使用权重量化对 LLMs 添加数字水印

Watermarking LLMs with Weight Quantization

Although Large Language Models (LLMs) represent a revolution in the way we
interact with computers, allowing the construction of complex questions and the
ability to reason over a sequence of statements, their use is restricted due to
the need for dedicated hardware for execution. In this study, we evaluate the
performance of LLMs based on the 7 and 13 billion LLaMA models, subjected to a
quantization process and run on home hardware. The models considered were
Alpaca, Koala, and Vicuna. To evaluate the effectiveness of these models, we
developed a database containing 1,006 questions from the ENEM (Brazilian
National Secondary School Exam). Our analysis revealed that the best performing
models achieved an accuracy of approximately 46% for the original texts of the
Portuguese questions and 49% on their English translations. In addition, we
evaluated the computational efficiency of the models by measuring the time
required for execution. On average, the 7 and 13 billion LLMs took
approximately 20 and 50 seconds, respectively, to process the queries on a
machine equipped with an AMD Ryzen 5 3600x processor

对 7 和 13 亿规模的大型语言模型 (Large Language Models, LLMs) 进行了性能评估，经过量化处理后在家用硬件上运行。通过包含 1,006 个巴西国家中学考试 (ENEM) 问题的数据库进行模型效果评估，最佳模型对原文葡萄牙语问题和其英文翻译的准确性分别达到约 46% 和 49%。此外，通过执行时间衡量模型的计算效率，7 和 13 亿规模的 LLMs 分别在装备 AMD Ryzen 5 3600x 处理器的机器上平均需要大约 20 秒和 50 秒来处理查询。