Language generation models have been an increasingly powerful enabler for
many applications. Many such models offer free or affordable API access, which
makes them potentially vulnerable to model extraction attacks through
distillation. To protect intellectual property (IP) and ensure fair use of
these models, various techniques such as lexical watermarking and synonym
replacement have been proposed. However, these methods can be nullified by
obvious countermeasures such as "synonym randomization". To address this issue,
we propose GINSEW, a novel method to protect text generation models from being
stolen through distillation. The key idea of our method is to inject secret
signals into the probability vector of the decoding steps for each target
token. We can then detect the secret message by probing a suspect model to tell
if it is distilled from the protected one. Experimental results show that
GINSEW can effectively identify instances of IP infringement with minimal
impact on the generation quality of protected APIs. Our method demonstrates an
absolute improvement of 19 to 29 points on mean average precision (mAP) in
detecting suspects compared to previous methods against watermark removal
attacks.

提出 GINSEW 方法以保护文本生成模型免受窃取，通过在目标令牌的解码步骤的概率向量中注入秘密信号，并在疑犯模型中检测秘密信息，GINSEW 可以有效地识别知识产权侵权的实例，表现出比之前的方法对抗去除水印攻击有 19 至 29 点的平均精度提升。