BriefGPT.xyz
Sep, 2021
基于Transformer网络的梯度自适应辅助任务选择方法——GradTS
GradTS: A Gradient-Based Automatic Auxiliary Task Selection Method Based on Transformer Networks
HTML
PDF
Weicheng Ma, Renze Lou, Kai Zhang, Lili Wang, Soroush Vosoughi
TL;DR
本研究介绍了基于Transformer的梯度计算的自动辅助任务选择方法GradTS,在八种自然语言理解任务中将MT-DNN的效果从0.33%提升到17.93%,且时间更节约且具有普适性。
Abstract
A key problem in
multi-task learning
(MTL) research is how to select high-quality auxiliary tasks automatically. This paper presents
gradts
, an automatic
→