The convergence speed of stochastic gradient descent (SGD) can be improved by actively selecting mini-batches. We explore sampling schemes where similar data points are less likely to be selected in the same mini-batch. In particular, we prove that such repulsive sampling schemes lowers the variance of the gradient estimator. This generalizes recent work on using Determinantal Point Processes (DPPs) for mini-batch diversification (Zhang et al., 2017) to the broader class of repulsive point processes. We first show that the phenomenon of variance reduction by diversified sampling generalizes in particular to non-stationary point processes. We then show that other point processes may be computationally much more efficient than DPPs. In particular, we propose and investigate Poisson Disk sampling---frequently encountered in the computer graphics community---for this task. We show empirically that our approach improves over standard SGD both in terms of convergence speed as well as final model performance.

本文探讨了在随机梯度下降中选择mini-batch的方法，证明了使用斥力采样来选择mini-batch可以降低梯度估计的方差，提出并研究实现 Poisson Disk 采样的可行性，并通过实验证明了这种方法在模型性能和收敛速度方面都要优于标准随机梯度下降。

利用斥力点过程实现主动小批量抽样