Model explanations improve the transparency of black-box machine learning
(ML) models and their decisions; however, they can also be exploited to carry
out privacy threats such as membership inference attacks (MIA). Existing works
have only analyzed MIA in a single "what if" interaction scenario between an
adversary and the target ML model; thus, it does not discern the factors
impacting the capabilities of an adversary in launching MIA in repeated
interaction settings. Additionally, these works rely on assumptions about the
adversary's knowledge of the target model's structure and, thus, do not
guarantee the optimality of the predefined threshold required to distinguish
the members from non-members. In this paper, we delve into the domain of
explanation-based threshold attacks, where the adversary endeavors to carry out
MIA attacks by leveraging the variance of explanations through iterative
interactions with the system comprising of the target ML model and its
corresponding explanation method. We model such interactions by employing a
continuous-time stochastic signaling game framework. In our framework, an
adversary plays a stopping game, interacting with the system (having imperfect
information about the type of an adversary, i.e., honest or malicious) to
obtain explanation variance information and computing an optimal threshold to
determine the membership of a datapoint accurately. First, we propose a sound
mathematical formulation to prove that such an optimal threshold exists, which
can be used to launch MIA. Then, we characterize the conditions under which a
unique Markov perfect equilibrium (or steady state) exists in this dynamic
system. By means of a comprehensive set of simulations of the proposed game
model, we assess different factors that can impact the capability of an
adversary to launch MIA in such repeated interaction settings.

黑盒机器学习模型的透明性可以通过模型解释来提高，然而这也会被利用来进行会员推断攻击。本文以持续时间随机信号博弈框架为基础，研究了基于解释的阈值攻击，分析了对于一个主动攻击者在不断交互情景下发起会员推断攻击的因素，并提出了数学模型来证明存在一个可以用于发起攻击的最佳阈值。