Transformer-based pre-training models like BERT have achieved remarkable
performance in many natural language processing tasks.However, these models are
both computation and memory expensive, hindering their deployment to
resource-constrained devices. In this work, we propose TernaryBERT, which
ternarizes the weights in a fine-tuned BERT model. Specifically, we use both
approximation-based and loss-aware ternarization methods and empirically
investigate the ternarization granularity of different parts of BERT. Moreover,
to reduce the accuracy degradation caused by the lower capacity of low bits, we
leverage the knowledge distillation technique in the training process.
Experiments on the GLUE benchmark and SQuAD show that our proposed TernaryBERT
outperforms the other BERT quantization methods, and even achieves comparable
performance as the full-precision model while being 14.9x smaller.

本文提出 TernaryBERT，在 fine-tuned BERT 模型中采用了三元权重化，通过近似法和损失感知方法研究了 BERT 不同部分的三元化粒度，并在训练过程中利用知识蒸馏技术来减少低位容量引起的精度下降，实验证明 TernaryBERT 不仅优于其他 BERT 量化方法，而且在 14.9 倍的情况下与全精度模型性能相当。