Prior works have attempted to build private inference frameworks for
transformer-based large language models (LLMs) in a server-client setting,
where the server holds the model parameters and the client inputs the private
data for inference. However, these frameworks impose significant overhead when
the private inputs are forward propagated through the original LLMs. In this
paper, we show that substituting the computation- and communication-heavy
operators in the transformer architecture with privacy-computing friendly
approximations can greatly reduce the private inference costs with minor impact
on model performance. Compared to the state-of-the-art Iron (NeurIPS 2022), our
privacy-computing friendly model inference pipeline achieves a $5\times$
acceleration in computation and an 80\% reduction in communication overhead,
while retaining nearly identical accuracy.

本文展示了将 transformer 模型中的运算和通信重负荷的操作替换成隐私计算友好的近似可以大大降低私有推理成本，并比先前最先进的 Iron（NeurIPS 2022）获得了 5 倍计算加速和 80% 的通信负担减少，同时保持几乎相同的准确性。