Ganqu Cui, Wentao Li, Ning Ding, Longtao Huang, Zhiyuan Liu...
TL;DR本研究提出 Decoder Tuning 方法,通过优化解码器网络来适应具有冻结参数的预训练模型,并只需要一个 API 查询,可以实现一千倍的加速。
Abstract
With the evergrowing sizes of pre-trained models (PTMs), it has been an
emerging practice to only provide the inference APIs for users, namely
model-as-a-service (MaaS) setting. To adapt PTMs with model parameter