Pre-trained models have achieved excellent performance on the dialogue task.
However, for the continual increase of online chit-chat scenarios, directly
fine-tuning these models for each of the new tasks not only explodes the
capacity of the dialogue system on the embedded devices but also causes
knowledge forgetting on pre-trained models and knowledge interference among
diverse dialogue tasks. In this work, we propose a hierarchical inductive
transfer framework to learn and deploy the dialogue skills continually and
efficiently. First, we introduce the adapter module into pre-trained models for
learning new dialogue tasks. As the only trainable module, it is beneficial for
the dialogue system on the embedded devices to acquire new dialogue skills with
negligible additional parameters. Then, for alleviating knowledge interference
between tasks yet benefiting the regularization between them, we further design
hierarchical inductive transfer that enables new tasks to use general knowledge
in the base adapter without being misled by diverse knowledge in task-specific
adapters. Empirical evaluation and analysis indicate that our framework obtains
comparable performance under deployment-friendly model capacity.

本文提出了一种基于递归传递的框架，通过引入适配器模块和迁移学习来解决在线聊天场景中为每个新任务调整模型导致容量过大及知识忘却问题。实验结果表明，在部署友好型模型容量下，我们的框架表现出可比较的性能

层级归纳迁移学习在持续对话学习中的应用

Hierarchical Inductive Transfer for Continual Dialogue Learning

We address the task of assessing discourse coherence, an aspect of text
quality that is essential for many NLP tasks, such as summarization and
language assessment. We propose a hierarchical neural network trained in a
multi-task fashion that learns to predict a document-level coherence score (at
the network's top layers) along with word-level grammatical roles (at the
bottom layers), taking advantage of inductive transfer between the two tasks.
We assess the extent to which our framework generalizes to different domains
and prediction tasks, and demonstrate its effectiveness not only on standard
binary evaluation coherence tasks, but also on real-world tasks involving the
prediction of varying degrees of coherence, achieving a new state of the art.

本研究提出一种层次神经网络模型，通过多任务学习，同时预测文档层次的信息连贯得分和词级别的语法角色，利用两个任务之间的归纳转移，提高模型泛化能力，并在不同任务中达到了新的最优表现。

一种用于连贯性建模的多任务学习方法

Multi-Task Learning for Coherence Modeling

An important goal of computer vision is to build systems that learn visual
representations over time that can be applied to many tasks. In this paper, we
investigate a vision-language embedding as a core representation and show that
it leads to better cross-task transfer than standard multi-task learning. In
particular, the task of visual recognition is aligned to the task of visual
question answering by forcing each to use the same word-region embeddings. We
show this leads to greater inductive transfer from recognition to VQA than
standard multitask learning. Visual recognition also improves, especially for
categories that have relatively few recognition training labels but appear
often in the VQA setting. Thus, our paper takes a small step towards creating
more general vision systems by showing the benefit of interpretable, flexible,
and trainable core representations.

本文研究视觉特征的表征方式，并探讨了通过将视觉识别任务和视觉问答任务各自对应的语言表示方式建立联系，以实现更好的跨任务数据迁移的方法。结果表明，这种方法比常规的多任务学习方法更有效，并且能够大幅提高在视觉识别任务和视觉问答任务上的准确率。