Recently, Deep Convolutional Neural Networks (DCNNs) including the ResNet-20
architecture have been privately evaluated on encrypted, low-resolution data
with the Residue-Number-System Cheon-Kim-Kim-Song (RNS-CKKS) homomorphic
encryption scheme. We extend methods for evaluating DCNNs on images with larger
dimensions and many channels, beyond what can be stored in single ciphertexts.
Additionally, we simplify and improve the efficiency of the recently introduced
multiplexed image format, demonstrating that homomorphic evaluation can work
with standard, row-major matrix packing and results in encrypted inference time
speedups by $4.6-6.5\times$. We also show how existing DCNN models can be
regularized during the training process to further improve efficiency and
accuracy. These techniques are applied to homomorphically evaluate a DCNN with
high accuracy on the high-resolution ImageNet dataset for the first time,
achieving $80.2\%$ top-1 accuracy. We also achieve the highest reported
accuracy of homomorphically evaluated CNNs on the CIFAR-10 dataset of $98.3\%$.

本文介绍了一种基于 RNS-CKKS 同态加密方案的深度卷积神经网络（DCNNs）的加密评估方法，通过对图像进行简化且高效的同态评估，获得了高精度对高分辨率 ImageNet 数据集的评估结果，同时在 CIFAR-10 数据集上获得了最高的同态评估的 CNN 准确性。

基于分片的密文同态加密数据的高分辨率卷积神经网络

High-Resolution Convolutional Neural Networks on Homomorphically  Encrypted Data via Sharding Ciphertexts

Magnitude pruning is a common, effective technique to identify sparse
subnetworks at little cost to accuracy. In this work, we ask whether a
particular architecture's accuracy-sparsity tradeoff can be improved by
combining pruning information across multiple runs of training. From a shared
ResNet-20 initialization, we train several network copies (\emph{siblings}) to
completion using different SGD data orders on CIFAR-10. While the siblings'
pruning masks are naively not much more similar than chance, starting sibling
training after a few epochs of shared pretraining significantly increases
pruning overlap. We then choose a subnetwork by either (1) taking all weights
that survive pruning in any sibling (mask union), or (2) taking only the
weights that survive pruning across all siblings (mask intersection). The
resulting subnetwork is retrained. Strikingly, we find that union and
intersection masks perform very similarly. Both methods match the
accuracy-sparsity tradeoffs of the one-shot magnitude pruning baseline, even
when we combine masks from up to $k = 10$ siblings.

本文研究了采用多轮训练的方式，是否能够提高特定体系结构的准确度与稀疏性之间的平衡，并在共享的 ResNet-20 初始化的基础上使用不同的 SGD 数据顺序训练多个网络副本来实现。我们的实验结果显示，采用多轮训练的思路与一次性的基于幅度裁剪的方法相比并无差异。