To support various applications, business owners often seek the customized
models that are obtained by fine-tuning a pre-trained LLM through the API
provided by LLM owners or cloud servers. However, this process carries a
substantial risk of model misuse, potentially resulting in severe economic
consequences for business owners. Thus, safeguarding the copyright of these
customized models during LLM fine-tuning has become an urgent practical
requirement, but there are limited existing solutions to provide such
protection. To tackle this pressing issue, we propose a novel watermarking
approach named "Double-I watermark". Specifically, based on the instruct-tuning
data, two types of backdoor data paradigms are introduced with trigger in the
instruction and the input, respectively. By leveraging LLM's learning
capability to incorporate customized backdoor samples into the dataset, the
proposed approach effectively injects specific watermarking information into
the customized model during fine-tuning, which makes it easy to inject and
verify watermarks in commercial scenarios. We evaluate the proposed "Double-I
watermark" under various fine-tuning methods, demonstrating its harmlessness,
robustness, uniqueness, imperceptibility, and validity through both theoretical
analysis and experimental verification.

为了保护商业定制模型的版权，在 LMM fine-tuning 过程中引入了一种名为 “Double-I watermark” 的新水印方法，通过两种类型的后门数据范例，有效地将特定的水印信息注入商业定制模型，并在各种 fine-tuning 方法下进行了评估。

双 I 水印：保护 LLM 微调模型的版权

Double-I Watermark: Protecting Model Copyright for LLM Fine-tuning

Image-text contrastive learning models such as CLIP have demonstrated strong
task transfer ability. The high generality and usability of these visual models
is achieved via a web-scale data collection process to ensure broad concept
coverage, followed by expensive pre-training to feed all the knowledge into
model weights. Alternatively, we propose REACT, REtrieval-Augmented
CusTomization, a framework to acquire the relevant web knowledge to build
customized visual models for target domains. We retrieve the most relevant
image-text pairs (~3% of CLIP pre-training data) from the web-scale database as
external knowledge, and propose to customize the model by only training new
modualized blocks while freezing all the original weights. The effectiveness of
REACT is demonstrated via extensive experiments on classification, retrieval,
detection and segmentation tasks, including zero, few, and full-shot settings.
Particularly, on the zero-shot classification task, compared with CLIP, it
achieves up to 5.4% improvement on ImageNet and 3.7% on the ELEVATER benchmark
(20 datasets).

提出一种名为 REACT 的框架用于获取相关网络知识以构建目标领域的定制视觉模型。该框架通过检索与预训练数据集最相关的图像 - 文本对作为外部知识源，并仅训练新的模块化块，冻结所有原始权重，证明了其在零、少、全样本设置下显著提高了分类、检索、检测和分割任务的成功率。