Protecting user privacy is a major concern for many machine learning systems
that are deployed at scale and collect from a diverse set of population. One
way to address this concern is by collecting and releasing data labels in an
aggregated manner so that the information about a single user is potentially
combined with others. In this paper, we explore the possibility of training
machine learning models with aggregated data labels, rather than individual
labels. Specifically, we consider two natural aggregation procedures suggested
by practitioners: curated bags where the data points are grouped based on
common features and random bags where the data points are grouped randomly in
bag of similar sizes. For the curated bag setting and for a broad range of loss
functions, we show that we can perform gradient-based learning without any
degradation in performance that may result from aggregating data. Our method is
based on the observation that the sum of the gradients of the loss function on
individual data examples in a curated bag can be computed from the aggregate
label without the need for individual labels. For the random bag setting, we
provide a generalization risk bound based on the Rademacher complexity of the
hypothesis class and show how empirical risk minimization can be regularized to
achieve the smallest risk bound. In fact, in the random bag setting, there is a
trade-off between size of the bag and the achievable error rate as our bound
indicates. Finally, we conduct a careful empirical study to confirm our
theoretical findings. In particular, our results suggest that aggregate
learning can be an effective method for preserving user privacy while
maintaining model accuracy.

本文探讨了使用聚合的数据标签训练机器学习模型的可行性，并针对策展袋和随机袋进行了两种自然聚合过程的研究。在策展袋设置下，通过使用聚合标签计算损失函数的梯度，证明我们可以实现不降低性能的基于梯度的训练。在随机袋设置下，基于 Rademacher 复杂度提供了一种广义风险边界，并展示了如何通过经验风险最小化进行正则化以实现最小风险界限，我们的实验结果表明，聚合学习可以有效地保护用户隐私并保持模型精度。