Transfer learning refers to the transfer of knowledge or information from a
relevant source task to a target task. However, most existing works assume both
tasks are sampled from a stationary task distribution, thereby leading to the
sub-optimal performance for dynamic tasks drawn from a non-stationary task
distribution in real scenarios. To bridge this gap, in this paper, we study a
more realistic and challenging transfer learning setting with dynamic tasks,
i.e., source and target tasks are continuously evolving over time. We
theoretically show that the expected error on the dynamic target task can be
tightly bounded in terms of source knowledge and consecutive distribution
discrepancy across tasks. This result motivates us to propose a generic
meta-learning framework L2E for modeling the knowledge transferability on
dynamic tasks. It is centered around a task-guided meta-learning problem with a
group of meta-pairs of tasks, based on which we are able to learn the prior
model initialization for fast adaptation on the newest target task. L2E enjoys
the following properties: (1) effective knowledge transferability across
dynamic tasks; (2) fast adaptation to the new target task; (3) mitigation of
catastrophic forgetting on historical target tasks; and (4) flexibility in
incorporating any existing static transfer learning algorithms. Extensive
experiments on various image data sets demonstrate the effectiveness of the
proposed L2E framework.

本文研究了一种更为现实和具有挑战性的转移学习设置，其中源任务和目标任务在时间上不断变化，我们提出了一个基于元学习的通用框架 L2E，它能够有效地在动态任务之间传递知识、快速适应新目标任务、减轻历史目标任务的灾难性遗忘，并且可灵活地结合任何现有的静态转移学习算法，验证实验表明该框架的有效性。

一种统一的元学习框架用于动态迁移学习

A Unified Meta-Learning Framework for Dynamic Transfer Learning

Developments in reinforcement learning (RL) have allowed algorithms to
achieve impressive performance in highly complex, but largely static problems.
In contrast, biological learning seems to value efficiency of adaptation to a
constantly-changing world. Here we build on a recently-proposed neuronal
learning rule that assumes each neuron can optimize its energy balance by
predicting its own future activity. That assumption leads to a neuronal
learning rule that uses presynaptic input to modulate prediction error. We
argue that an analogous RL rule would use action probability to modulate reward
prediction error. This modulation makes the agent more sensitive to negative
experiences, and more careful in forming preferences. We embed the proposed
rule in both tabular and deep-Q-network RL algorithms, and find that it
outperforms conventional algorithms in simple, but highly-dynamic tasks. We
suggest that the new rule encapsulates a core principle of biological
intelligence; an important component for allowing algorithms to adapt to change
in a human-like way.

研究提出了一种新的神经元学习规则，它使用突触前输入来调制预测误差，将其嵌入表格和深度 Q 网络强化学习算法中，可以在简单而高动态的任务中胜过传统算法，这提出了一种新的生物智能核心原则。