Training machine learning models on data from multiple entities without
direct data sharing can unlock applications otherwise hindered by business,
legal, or ethical constraints. In this work, we design and implement new
privacy-preserving machine learning protocols for logistic regression and
neural network models. We adopt a two-server model where data owners
secret-share their data between two servers that train and evaluate the model
on the joint data. A significant source of inefficiency and inaccuracy in
existing methods arises from using Yao's garbled circuits to compute non-linear
activation functions. We propose new methods for computing non-linear functions
based on secret-shared lookup tables, offering both computational efficiency
and improved accuracy.
Beyond introducing leakage-free techniques, we initiate the exploration of
relaxed security measures for privacy-preserving machine learning. Instead of
claiming that the servers gain no knowledge during the computation, we contend
that while some information is revealed about access patterns to lookup tables,
it maintains epsilon-dX-privacy. Leveraging this relaxation significantly
reduces the computational resources needed for training. We present new
cryptographic protocols tailored to this relaxed security paradigm and define
and analyze the leakage. Our evaluations show that our logistic regression
protocol is up to 9x faster, and the neural network training is up to 688x
faster than SecureML. Notably, our neural network achieves an accuracy of 96.6%
on MNIST in 15 epochs, outperforming prior benchmarks that capped at 93.4%
using the same architecture.

训练具有多个实体的机器学习模型，并且在没有直接数据共享的情况下，可以解决由于业务、法律或道德限制而受阻的应用。本文设计和实施了新的隐私保护机器学习协议，用于逻辑回归和神经网络模型。我们采用了两个服务器模型，在数据所有者之间进行数据密钥共享，由这两个服务器进行联合数据的模型训练和评估。现有方法中存在的低效和不准确的重要方法是使用 Yao 噪声电路来计算非线性激活函数的来源。我们提出了基于密钥共享的查找表的计算非线性函数的新方法，既提供了计算效率，又提高了准确性。此外，我们还提出了一种针对隐私保护机器学习的放宽安全措施的探索。我们认为，在计算过程中，虽然有关于查找表访问模式的一些信息被透露，但它仍然保持 epsilon-dX 隐私。利用这种放松可显著减少训练所需的计算资源。我们提出了针对这种放松安全范式量身定制的加密协议，并定义和分析泄漏。我们的评估结果表明，我们的逻辑回归协议比 SecureML 快 9 倍，神经网络训练速度比 SecureML 快 688 倍。值得注意的是，我们的神经网络在 15 个周期内在 MNIST 数据集上实现了 96.6% 的准确率，超过了以往在相同架构下准确率为 93.4% 的基准。