Membership Inference Attacks (MIAs) are widely used to evaluate the
propensity of a machine learning (ML) model to memorize an individual record
and the privacy risk releasing the model poses. MIAs are commonly evaluated
similarly to ML models: the MIA is performed on a test set of models trained on
datasets unseen during training, which are sampled from a larger pool,
$D_{eval}$. The MIA is evaluated across all datasets in this test set, and is
thus evaluated across the distribution of samples from $D_{eval}$. While this
was a natural extension of ML evaluation to MIAs, recent work has shown that a
record's risk heavily depends on its specific dataset. For example, outliers
are particularly vulnerable, yet an outlier in one dataset may not be one in
another. The sources of randomness currently used to evaluate MIAs may thus
lead to inaccurate individual privacy risk estimates. We propose a new,
specific evaluation setup for MIAs against ML models, using weight
initialization as the sole source of randomness. This allows us to accurately
evaluate the risk associated with the release of a model trained on a specific
dataset. Using SOTA MIAs, we empirically show that the risk estimates given by
the current setup lead to many records being misclassified as low risk. We
derive theoretical results which, combined with empirical evidence, suggest
that the risk calculated in the current setup is an average of the risks
specific to each sampled dataset, validating our use of weight initialization
as the only source of randomness. Finally, we consider an MIA with a stronger
adversary leveraging information about the target dataset to infer membership.
Taken together, our results show that current MIA evaluation is averaging the
risk across datasets leading to inaccurate risk estimates, and the risk posed
by attacks leveraging information about the target dataset to be potentially
underestimated.

使用权重初始化作为唯一的随机源，我们提出了一种新的、特定的用于对 ML 模型的成员识别攻击（MIAs）进行评估的设置，从而准确评估了与特定数据集训练的模型的发布相关的风险。