We present SkillNet-NLG, a sparsely activated approach that handles many natural language generation tasks with one model. Different from traditional dense models that always activate all the parameters, SkillNet-NLG selectively activates relevant parts of the parameters to accomplish a task, where the relevance is controlled by a set of predefined skills. The strength of such model design is that it provides an opportunity to precisely adapt relevant skills to learn new tasks effectively. We evaluate on Chinese natural language generation tasks. Results show that, with only one model file, SkillNet-NLG outperforms previous best performance methods on four of five tasks. SkillNet-NLG performs better than two multi-task learning baselines (a dense model and a Mixture-of-Expert model) and achieves comparable performance to task-specific models. Lastly, SkillNet-NLG surpasses baseline systems when being adapted to new tasks.

我们介绍了SkillNet-NLG，一种稀疏激活的方法，可以处理多种自然语言生成任务。该方法与传统的密集模型不同，SkillNet-NLG仅选择激活与任务相关的参数，这由一组预定义的技能控制。实验结果表明，SkillNet-NLG可以胜任五项任务中的四项，并且优于两种多任务模型和任务特定模型，同时也在适应新任务时表现出色。

SkillNet-NLG: 一种稀疏激活法的通用自然语言生成器