Machine learning models are known to memorize private data to reduce their
training loss, which can be inadvertently exploited by privacy attacks such as
model inversion and membership inference. To protect against these attacks,
differential privacy (DP) has become the de facto standard for
privacy-preserving machine learning, particularly those popular training
algorithms using stochastic gradient descent, such as DPSGD. Nonetheless, DPSGD
still suffers from severe utility loss due to its slow convergence. This is
partially caused by the random sampling, which brings bias and variance to the
gradient, and partially by the Gaussian noise, which leads to fluctuation of
gradient updates.
Our key idea to address these issues is to apply selective updates to the
model training, while discarding those useless or even harmful updates.
Motivated by this, this paper proposes DPSUR, a Differentially Private training
framework based on Selective Updates and Release, where the gradient from each
iteration is evaluated based on a validation test, and only those updates
leading to convergence are applied to the model. As such, DPSUR ensures the
training in the right direction and thus can achieve faster convergence than
DPSGD. The main challenges lie in two aspects -- privacy concerns arising from
gradient evaluation, and gradient selection strategy for model update. To
address the challenges, DPSUR introduces a clipping strategy for update
randomization and a threshold mechanism for gradient selection. Experiments
conducted on MNIST, FMNIST, CIFAR-10, and IMDB datasets show that DPSUR
significantly outperforms previous works in terms of convergence speed and
model utility.

DPSUR 是基于选择性更新和发布的差分隐私训练框架，通过评估每次迭代的梯度并只使用导致收敛的更新，实现了比 DPSGD 更快的收敛速度，实验结果表明 DPSUR 在收敛速度和模型效用方面明显优于之前的研究成果。

DPSUR: 通过选择性更新和发布加速差分隐私随机梯度下降

DPSUR: Accelerating Differentially Private Stochastic Gradient Descent  Using Selective Update and Release

Dynamic Bayesian networks (DBNs) are a general model for stochastic processes
with partially observed states. Belief filtering in DBNs is the task of
inferring the belief state (i.e. the probability distribution over process
states) based on incomplete and uncertain observations. In this article, we
explore the idea of accelerating the filtering task by automatically exploiting
causality in the process. We consider a specific type of causal relation,
called passivity, which pertains to how state variables cause changes in other
variables. We present the Passivity-based Selective Belief Filtering (PSBF)
method, which maintains a factored belief representation and exploits passivity
to perform selective updates over the belief factors. PSBF is evaluated in both
synthetic processes and a simulated multi-robot warehouse, where it
outperformed alternative filtering methods by exploiting passivity.

该研究探索了利用因果关系加速贝叶斯网络中的信念过滤任务的想法，并提出了一种基于被动性的选择性信念过滤方法（PSBF），该方法在维护因果关系的情况下，在虚拟进程和模拟多机器人仓库中均优于传统滤波方法。