Ad relevance modeling plays a critical role in online advertising systems
including Microsoft Bing. To leverage powerful transformers like BERT in this
low-latency setting, many existing approaches perform ad-side computations
offline. While efficient, these approaches are unable to serve cold start ads,
resulting in poor relevance predictions for such ads. This work aims to design
a new, low-latency BERT via structured pruning to empower real-time online
inference for cold start ads relevance on a CPU platform. Our challenge is that
previous methods typically prune all layers of the transformer to a high,
uniform sparsity, thereby producing models which cannot achieve satisfactory
inference speed with an acceptable accuracy.
In this paper, we propose SwiftPruner - an efficient framework that leverages
evolution-based search to automatically find the best-performing layer-wise
sparse BERT model under the desired latency constraint. Different from existing
evolution algorithms that conduct random mutations, we propose a reinforced
mutator with a latency-aware multi-objective reward to conduct better mutations
for efficiently searching the large space of layer-wise sparse models.
Extensive experiments demonstrate that our method consistently achieves higher
ROC AUC and lower latency than the uniform sparse baseline and state-of-the-art
search methods. Remarkably, under our latency requirement of 1900us on CPU,
SwiftPruner achieves a 0.86% higher AUC than the state-of-the-art uniform
sparse baseline for BERT-Mini on a large scale real-world dataset. Online A/B
testing shows that our model also achieves a significant 11.7% cut in the ratio
of defective cold start ads with satisfactory real-time serving latency.

本文提出了 SwiftPruner，它是一个在线广告关键词预测的低延迟模型，采用结构剪枝的方法，利用演化搜索算法进行自动调整。通过多目标奖励来指导基因学习，优化层次稀疏模型的性能，该模型不仅在模拟测试中比其他模型具有更高的性能，而且在实际测试中更有效地降低了冷启动广告错误率。