Machine learning involves expensive data collection and training procedures.
Model owners may be concerned that valuable intellectual property can be leaked
if adversaries mount model extraction attacks. As it is difficult to defend
against model extraction without sacrificing significant prediction accuracy,
watermarking instead leverages unused model capacity to have the model overfit
to outlier input-output pairs. Such pairs are watermarks, which are not sampled
from the task distribution and are only known to the defender. The defender
then demonstrates knowledge of the input-output pairs to claim ownership of the
model at inference. The effectiveness of watermarks remains limited because
they are distinct from the task distribution and can thus be easily removed
through compression or other forms of knowledge transfer.
We introduce Entangled Watermarking Embeddings (EWE). Our approach encourages
the model to learn features for classifying data that is sampled from the task
distribution and data that encodes watermarks. An adversary attempting to
remove watermarks that are entangled with legitimate data is also forced to
sacrifice performance on legitimate data. Experiments on MNIST, Fashion-MNIST,
CIFAR-10, and Speech Commands validate that the defender can claim model
ownership with 95\% confidence with less than 100 queries to the stolen copy,
at a modest cost below 0.81 percentage points on average in the defended
model's performance.

本文提出一种名为 EWE 的恒定水印嵌入技术，该技术将水印紧密嵌入到训练任务中，促使模型同时学习正常输入和此类水印，并在测试时表现优异，可实现在 100 次以下查询成功地断言模型属于自己。