In this demonstration, we present an efficient BERT-based multi-task (MT)
framework that is particularly suitable for iterative and incremental
development of the tasks. The proposed framework is based on the idea of
partial fine-tuning, i.e. only fine-tune some top layers of BERT while keep the
other layers frozen. For each task, we train independently a single-task (ST)
model using partial fine-tuning. Then we compress the task-specific layers in
each ST model using knowledge distillation. Those compressed ST models are
finally merged into one MT model so that the frozen layers of the former are
shared across the tasks. We exemplify our approach on eight GLUE tasks,
demonstrating that it is able to achieve both strong performance and
efficiency. We have implemented our method in the utterance understanding
system of XiaoAI, a commercial AI assistant developed by Xiaomi. We estimate
that our model reduces the overall serving cost by 86%.

本文提出一种基于 BERT 的多任务框架，采用部分微调，单任务模型知识蒸馏和模型压缩技术，使得各个子任务的共享部分得以共用，从而实现了低成本、高性能的实时任务处理。该框架在小米智能语音助手中得到了应用，整个系统处理成本降低了 86%。