We present MT-DNN, an open-source natural language understanding (NLU)
toolkit that makes it easy for researchers and developers to train customized
deep learning models. Built upon PyTorch and Transformers, MT-DNN is designed
to facilitate rapid customization for a broad spectrum of NLU tasks, using a
variety of objectives (classification, regression, structured prediction) and
text encoders (e.g., RNNs, BERT, RoBERTa, UniLM). A unique feature of MT-DNN is
its built-in support for robust and transferable learning using the adversarial
multi-task learning paradigm. To enable efficient production deployment, MT-DNN
supports multi-task knowledge distillation, which can substantially compress a
deep neural model without significant performance drop. We demonstrate the
effectiveness of MT-DNN on a wide range of NLU applications across general and
biomedical domains. The software and pre-trained models will be publicly
available at this https URL

MT-DNN 是一种基于 PyTorch 和 Transformers 的自然语言理解工具包，支持深度学习模型的个性化训练，使用多种目标和文本编码器，并具有鲁棒性和可迁移性的多任务学习方法，支持多任务知识蒸馏。MT-DNN 已在一系列 NLU 应用中展示了其效果。

自然语言理解的多任务深度神经网络 Microsoft 工具包

The Microsoft Toolkit of Multi-Task Deep Neural Networks for Natural  Language Understanding

This paper describes our competing system to enter the MEDIQA-2019
competition. We use a multi-source transfer learning approach to transfer the
knowledge from MT-DNN and SciBERT to natural language understanding tasks in
the medical domain. For transfer learning fine-tuning, we use multi-task
learning on NLI, RQE and QA tasks on general and medical domains to improve
performance. The proposed methods are proved effective for natural language
understanding in the medical domain, and we rank the first place on the QA
task.

本文介绍了我们参加 MEDIQA-2019 比赛的系统，我们使用多源迁移学习方法，将 MT-DNN 和 SciBERT 中的知识应用于医疗领域的自然语言理解任务中。我们采用多任务学习技术，在通用和医疗领域上进行 NLI、RQE 和 QA 任务的微调，证明了所提出的方法在医疗领域的自然语言理解任务中有效，并在 QA 任务上获得了第一名。