The Vision Transformer (ViT) architecture has emerged as the backbone of
choice for state-of-the-art deep models for computer vision applications.
However, ViTs are ill-suited for private inference using secure multi-party
computation (MPC) protocols, due to the large number of non-polynomial
operations (self-attention, feed-forward rectifiers, layer normalization). We
propose PriViT, a gradient based algorithm to selectively "Taylorize"
nonlinearities in ViTs while maintaining their prediction accuracy. Our
algorithm is conceptually simple, easy to implement, and achieves improved
performance over existing approaches for designing MPC-friendly transformer
architectures in terms of achieving the Pareto frontier in latency-accuracy. We
confirm these improvements via experiments on several standard image
classification tasks. Public code is available at
this https URL

使用 PriViT 算法，通过温和改变 Vision Transformer 结构中的非线性变换，实现了在维持预测准确性的同时，适应安全多方计算（MPC）协议的隐私预测，相比于现有方案，PriViT 在延迟 - 准确性的 Pareto 最优曲线方面取得改进。