In many real-world applications, in particular due to recent developments in
the privacy landscape, training data may be aggregated to preserve the privacy
of sensitive training labels. In the learning from label proportions (LLP)
framework, the dataset is partitioned into bags of feature-vectors which are
available only with the sum of the labels per bag. A further restriction, which
we call learning from bag aggregates (LBA) is where instead of individual
feature-vectors, only the (possibly weighted) sum of the feature-vectors per
bag is available. We study whether such aggregation techniques can provide
privacy guarantees under the notion of label differential privacy (label-DP)
previously studied in for e.g. [Chaudhuri-Hsu'11, Ghazi et al.'21, Esfandiari
et al.'22].
It is easily seen that naive LBA and LLP do not provide label-DP. Our main
result however, shows that weighted LBA using iid Gaussian weights with $m$
randomly sampled disjoint $k$-sized bags is in fact $(\varepsilon,
\delta)$-label-DP for any $\varepsilon > 0$ with $\delta \approx
\exp(-\Omega(\sqrt{k}))$ assuming a lower bound on the linear-mse regression
loss. Further, this preserves the optimum over linear mse-regressors of bounded
norm to within $(1 \pm o(1))$-factor w.p. $\approx 1 - \exp(-\Omega(m))$. We
emphasize that no additive label noise is required.
The analogous weighted-LLP does not however admit label-DP. Nevertheless, we
show that if additive $N(0, 1)$ noise can be added to any constant fraction of
the instance labels, then the noisy weighted-LLP admits similar label-DP
guarantees without assumptions on the dataset, while preserving the utility of
Lipschitz-bounded neural mse-regression tasks.
Our work is the first to demonstrate that label-DP can be achieved by
randomly weighted aggregation for regression tasks, using no or little additive
noise.

本研究首次证明了在回归任务中，使用加权聚合方法（不需要或只需很小的附加噪声）可实现标签差分隐私（label-DP）。