BERT-based neural architectures have established themselves as popular
state-of-the-art baselines for many downstream NLP tasks. However, these
architectures are data-hungry and consume a lot of memory and energy, often
hindering their deployment in many real-time, resource-constrained
applications. Existing lighter versions of BERT (eg. DistilBERT and TinyBERT)
often cannot perform well on complex NLP tasks. More importantly, from a
designer's perspective, it is unclear what is the "right" BERT-based
architecture to use for a given NLP task that can strike the optimal trade-off
between the resources available and the minimum accuracy desired by the end
user. System engineers have to spend a lot of time conducting trial-and-error
experiments to find a suitable answer to this question. This paper presents an
exploratory study of BERT-based models under different resource constraints and
accuracy budgets to derive empirical observations about this resource/accuracy
trade-offs. Our findings can help designers to make informed choices among
alternative BERT-based architectures for embedded systems, thus saving
significant development time and effort.

通过对 BERT-based 模型在不同资源限制和准确度预算下的实证研究，发现一个最优资源和准确度平衡点，帮助设计者在替代的 BERT-based 架构中做出明智的选择，节省显著的开发时间和精力。

在资源受限嵌入式设备中部署 BERT NLP 模型的挑战探索

Exploring Challenges of Deploying BERT-based NLP Models in  Resource-Constrained Embedded Devices

This paper describes our system for SemEval-2021 Task 5 on Toxic Spans
Detection. We developed ensemble models using BERT-based neural architectures
and post-processing to combine tokens into spans. We evaluated several
pre-trained language models using various ensemble techniques for toxic span
identification and achieved sizable improvements over our baseline fine-tuned
BERT models. Finally, our system obtained a F1-score of 67.55% on test data.

本文描述了我们基于 BERT 神经架构和后处理方法的集成模型，在 SemEval-2021 任务 5 的毒性跨度检测中应用的系统。我们使用多种集成技术对多个预训练语言模型进行了评估，实现了相当大的改进，并在测试数据上获得了 67.55％的 F1 分数。