Label differential privacy (DP) is a framework that protects the privacy of
labels in training datasets, while the feature vectors are public. Existing
approaches protect the privacy of labels by flipping them randomly, and then
train a model to make the output approximate the privatized label. However, as
the number of classes $K$ increases, stronger randomization is needed, thus the
performances of these methods become significantly worse. In this paper, we
propose a vector approximation approach, which is easy to implement and
introduces little additional computational overhead. Instead of flipping each
label into a single scalar, our method converts each label into a random vector
with $K$ components, whose expectations reflect class conditional
probabilities. Intuitively, vector approximation retains more information than
scalar labels. A brief theoretical analysis shows that the performance of our
method only decays slightly with $K$. Finally, we conduct experiments on both
synthesized and real datasets, which validate our theoretical analysis as well
as the practical performance of our method.

我们提出了一种向量逼近方法，将每个标签转换成具有 K 个分量的随机向量，其期望值反映类别条件概率，该方法在理论上只略微下降，并通过实验证实了我们的理论分析和方法的实际性能。

利用向量逼近提高标签差分隐私的学习

Enhancing Learning with Label Differential Privacy by Vector  Approximation

This paper addresses the nearest neighbor search problem under inner product
similarity and introduces a compact code-based approach. The idea is to
approximate a vector using the composition of several elements selected from a
source dictionary and to represent this vector by a short code composed of the
indices of the selected elements. The inner product between a query vector and
a database vector is efficiently estimated from the query vector and the short
code of the database vector. We show the superior performance of the proposed
group $M$-selection algorithm that selects $M$ elements from $M$ source
dictionaries for vector approximation in terms of search accuracy and
efficiency for compact codes of the same length via theoretical and empirical
analysis. Experimental results on large-scale datasets ($1M$ and $1B$ SIFT
features, $1M$ linear models and Netflix) demonstrate the superiority of the
proposed approach.

该论文提出了一种基于紧凑代码的近似向量检索算法，通过选择源字典中的若干元素来近似一个向量，并将其表示为由所选元素的索引组成的短代码，从而实现查询向量和数据库向量间的内积相似度检索。通过理论分析和实验结果表明，提出的 $M$-selection 算法在紧凑代码相同长度的情况下，在检索精度和速度方面表现出卓越的性能。