With the yearning for deep learning democratization, there are increasing
demands to implement Transformer-based natural language processing (NLP) models
on resource-constrained devices for low-latency and high accuracy. Existing
BERT pruning methods require domain experts to heuristically handcraft
hyperparameters to strike a balance among model size, latency, and accuracy. In
this work, we propose AE-BERT, an automatic and efficient BERT pruning
framework with efficient evaluation to select a "good" sub-network candidate
(with high accuracy) given the overall pruning ratio constraints. Our proposed
method requires no human experts experience and achieves a better accuracy
performance on many NLP tasks. Our experimental results on General Language
Understanding Evaluation (GLUE) benchmark show that AE-BERT outperforms the
state-of-the-art (SOTA) hand-crafted pruning methods on BERT$_{\mathrm{BASE}}$.
On QNLI and RTE, we obtain 75\% and 42.8\% more overall pruning ratio while
achieving higher accuracy. On MRPC, we obtain a 4.6 higher score than the SOTA
at the same overall pruning ratio of 0.5. On STS-B, we can achieve a 40\%
higher pruning ratio with a very small loss in Spearman correlation compared to
SOTA hand-crafted pruning methods. Experimental results also show that after
model compression, the inference time of a single BERT$_{\mathrm{BASE}}$
encoder on Xilinx Alveo U200 FPGA board has a 1.83$\times$ speedup compared to
Intel(R) Xeon(R) Gold 5218 (2.30GHz) CPU, which shows the reasonableness of
deploying the proposed method generated subnets of BERT$_{\mathrm{BASE}}$ model
on computation restricted devices.

提出 AE-BERT，一种自动和高效的 BERT 剪枝框架，可在资源受限设备上实现基于 Transformer 自然语言处理模型并提高准确性。实验结果表明，在压缩模型后，与 CPU 相比使用 AE-BERT 产生的 BERT 基础编码器子网络可以在类似的总剪枝比率下获得更高的分数，并提高推理时间。