Model stealing aims at inferring a victim model's functionality at a fraction
of the original training cost. While the goal is clear, in practice the model's
architecture, weight dimension, and original training data can not be
determined exactly, leading to mutual uncertainty during stealing. In this
work, we explicitly tackle this uncertainty by generating multiple possible
networks and combining their predictions to improve the quality of the stolen
model. For this, we compare five popular uncertainty quantification models in a
model stealing task. Surprisingly, our results indicate that the considered
models only lead to marginal improvements in terms of label agreement (i.e.,
fidelity) to the stolen model. To find the cause of this, we inspect the
diversity of the model's prediction by looking at the prediction variance as a
function of training iterations. We realize that during training, the models
tend to have similar predictions, indicating that the network diversity we
wanted to leverage using uncertainty quantification models is not (high) enough
for improvements on the model stealing task.

本论文主要研究如何通过不确定性量化和提高网络多样性来提高模型盗取的效果，但实验结果表明所考虑的模型对于提高盗取模型的忠实度只有微小的改进，我们通过观察模型预测的多样性发现可能是因为训练期间，不同网络的预测趋于相似，这也许是使用不确定性量化模型不能够提高模型盗取的原因之一。