Transfer learning (TL) has been demonstrated to improve DNN model performance
when faced with a scarcity of training samples. However, the suitability of TL
as a solution to reduce vulnerability of overfitted DNNs to privacy attacks is
unexplored. A class of privacy attacks called membership inference attacks
(MIAs) aim to determine whether a given sample belongs to the training dataset
(member) or not (nonmember). We introduce Double-Dip, a systematic empirical
study investigating the use of TL (Stage-1) combined with randomization
(Stage-2) to thwart MIAs on overfitted DNNs without degrading classification
accuracy. Our study examines the roles of shared feature space and parameter
values between source and target models, number of frozen layers, and
complexity of pretrained models. We evaluate Double-Dip on three (Target,
Source) dataset paris: (i) (CIFAR-10, ImageNet), (ii) (GTSRB, ImageNet), (iii)
(CelebA, VGGFace2). We consider four publicly available pretrained DNNs: (a)
VGG-19, (b) ResNet-18, (c) Swin-T, and (d) FaceNet. Our experiments demonstrate
that Stage-1 reduces adversary success while also significantly increasing
classification accuracy of nonmembers against an adversary with either
white-box or black-box DNN model access, attempting to carry out SOTA
label-only MIAs. After Stage-2, success of an adversary carrying out a
label-only MIA is further reduced to near 50%, bringing it closer to a random
guess and showing the effectiveness of Double-Dip. Stage-2 of Double-Dip also
achieves lower ASR and higher classification accuracy than regularization and
differential privacy-based methods.

使用 Transfer Learning（TL）和随机化相结合的 Double-Dip 方法，通过降低过拟合 DNN 对隐私攻击的脆弱性，显著提高非成员的分类准确性，以及减少标签推理攻击的成功率。