Although deep neural networks have made tremendous progress in the area of
multimedia representation, training neural models requires a large amount of
data and time. It is well-known that utilizing trained models as initial
weights often achieves lower training error than neural networks that are not
pre-trained. A fine-tuning step helps to reduce both the computational cost and
improve performance. Therefore, sharing trained models has been very important
for the rapid progress of research and development. In addition, trained models
could be important assets for the owner(s) who trained them, hence we regard
trained models as intellectual property. In this paper, we propose a digital
watermarking technology for ownership authorization of deep neural networks.
First, we formulate a new problem: embedding watermarks into deep neural
networks. We also define requirements, embedding situations, and attack types
on watermarking in deep neural networks. Second, we propose a general framework
for embedding a watermark in model parameters, using a parameter regularizer.
Our approach does not impair the performance of networks into which a watermark
is placed because the watermark is embedded while training the host network.
Finally, we perform comprehensive experiments to reveal the potential of
watermarking deep neural networks as the basis of this new research effort. We
show that our framework can embed a watermark during the training of a deep
neural network from scratch, and during fine-tuning and distilling, without
impairing its performance. The embedded watermark does not disappear even after
fine-tuning or parameter pruning; the watermark remains complete even after 65%
of parameters are pruned.

本文提出了一种深度神经网络数字水印技术，可用于深度神经网络的所有权授权。我们定义了嵌入深度神经网络数字水印的要求、嵌入情况和攻击类型，并提出了一种嵌入参数规则器的通用框架，最后通过实验证明我们的框架可以在深度神经网络的训练中嵌入数字水印，并在精调和参数修剪后保留所有权编码。