The escalating focus on data privacy poses significant challenges for
collaborative neural network training, where data ownership and model
training/deployment responsibilities reside with distinct entities. Our
community has made substantial contributions to addressing this challenge,
proposing various approaches such as federated learning (FL) and
privacy-preserving machine learning based on cryptographic constructs like
homomorphic encryption (HE) and secure multiparty computation (MPC). However,
FL completely overlooks model privacy, and HE has limited extensibility
(confined to only one data provider). While the state-of-the-art MPC frameworks
provide reasonable throughput and simultaneously ensure model/data privacy,
they rely on a critical non-colluding assumption on the computing servers, and
relaxing this assumption is still an open problem.
In this paper, we present Pencil, the first private training framework for
collaborative learning that simultaneously offers data privacy, model privacy,
and extensibility to multiple data providers, without relying on the
non-colluding assumption. Our fundamental design principle is to construct the
n-party collaborative training protocol based on an efficient two-party
protocol, and meanwhile ensuring that switching to different data providers
during model training introduces no extra cost. We introduce several novel
cryptographic protocols to realize this design principle and conduct a rigorous
security and privacy analysis. Our comprehensive evaluations of Pencil
demonstrate that (i) models trained in plaintext and models trained privately
using Pencil exhibit nearly identical test accuracies; (ii) The training
overhead of Pencil is greatly reduced: Pencil achieves 10 ~ 260x higher
throughput and 2 orders of magnitude less communication than prior art; (iii)
Pencil is resilient against both existing and adaptive (white-box) attacks.

Pencil 是第一种为协作学习提供数据隐私、模型隐私和可扩展性的私有训练框架，无需依赖于非勾结假设。

Pencil: 私密且可扩展的协作学习，无需无合谋假设

Pencil: Private and Extensible Collaborative Learning without the  Non-Colluding Assumption

Multiparty computation approaches to secure neural network inference
traditionally rely on garbled circuits for securely executing nonlinear
activation functions. However, garbled circuits require excessive communication
between server and client, impose significant storage overheads, and incur
large runtime penalties. To eliminate these costs, we propose an alternative to
garbled circuits: Tabula, an algorithm based on secure lookup tables. Tabula
leverages neural networks' ability to be quantized and employs a secure lookup
table approach to efficiently, securely, and accurately compute neural network
nonlinear activation functions. Compared to garbled circuits with quantized
inputs, when computing individual nonlinear functions, our experiments show
Tabula uses between $35 \times$-$70 \times$ less communication, is over
$100\times$ faster, and uses a comparable amount of storage. This leads to
significant performance gains over garbled circuits with quantized inputs
during secure inference on neural networks: Tabula reduces overall
communication by up to $9 \times$ and achieves a speedup of up to $50 \times$,
while imposing comparable storage costs.

该研究通过采用安全查找表的算法 Tabula，代替使用加密电路进行神经网络推断所需的非线性激活函数加密计算，以此降低了通信、存储和运行成本，并在安全神经网络推断时实现了显著的性能提升。