Privacy-preserving neural networks have attracted increasing attention in
recent years, and various algorithms have been developed to keep the balance
between accuracy, computational complexity and information security from the
cryptographic view. This work takes a different view from the input data and
structure of neural networks. We decompose the input data (e.g., some images)
into sensitive and insensitive segments according to importance and privacy.
The sensitive segment includes some important and private information such as
human faces and we take strong homomorphic encryption to keep security, whereas
the insensitive one contains some background and we add perturbations. We
propose the bi-CryptoNets, i.e., plaintext and ciphertext branches, to deal
with two segments, respectively, and ciphertext branch could utilize the
information from plaintext branch by unidirectional connections. We adopt
knowledge distillation for our bi-CryptoNets by transferring representations
from a well-trained teacher neural network. Empirical studies show the
effectiveness and decrease of inference latency for our bi-CryptoNets.

该研究采用隐私保护神经网络中的不同方法，通过敏感和非敏感部分的分解以及密文和明文的分支，利用同态加密和知识蒸馏等技术提高信息安全性、降低推理延迟。

双加密网络：利用不同层面的隐私进行加密推理

Bi-CryptoNets: Leveraging Different-Level Privacy for Encrypted  Inference

Recently Homomorphic Encryption (HE) is used to implement Privacy-Preserving
Neural Networks (PPNNs) that perform inferences directly on encrypted data
without decryption. Prior PPNNs adopt mobile network architectures such as
SqueezeNet for smaller computing overhead, but we find na\"ively using mobile
network architectures for a PPNN does not necessarily achieve shorter inference
latency. Despite having less parameters, a mobile network architecture
typically introduces more layers and increases the HE multiplicative depth of a
PPNN, thereby prolonging its inference latency. In this paper, we propose a
\textbf{HE}-friendly privacy-preserving \textbf{M}obile neural n\textbf{ET}work
architecture, \textbf{HEMET}. Experimental results show that, compared to
state-of-the-art (SOTA) PPNNs, HEMET reduces the inference latency by
$59.3\%\sim 61.2\%$, and improves the inference accuracy by $0.4 \% \sim
0.5\%$.

本论文提出了一种 HEMET 的 HE 友好型隐私保护移动神经网络架构，相较于之前的 PPNNs，HEMET 可以降低推理延迟 59.3%~61.2% 并提高 0.4%~0.5% 的推理精度。