Although transfer learning is considered to be a milestone in deep
reinforcement learning, the mechanisms behind it are still poorly understood.
In particular, predicting if knowledge can be transferred between two given
tasks is still an unresolved problem. In this work, we explore the use of
network distillation as a feature extraction method to better understand the
context in which transfer can occur. Notably, we show that distillation does
not prevent knowledge transfer, including when transferring from multiple tasks
to a new one, and we compare these results with transfer without prior
distillation. We focus our work on the Atari benchmark due to the variability
between different games, but also to their similarities in terms of visual
features.

本文探索了使用网络蒸馏作为特征提取方法的转移学习，发现蒸馏不妨碍知识转移，包括从多个任务到一个新任务的转移，结果比没有使用蒸馏更优。

强化学习中神经巩固的迁移学习

On Neural Consolidation for Transfer in Reinforcement Learning

This paper theoretically investigates the following empirical phenomenon:
given a high-complexity network with poor generalization bounds, one can
distill it into a network with nearly identical predictions but low complexity
and vastly smaller generalization bounds. The main contribution is an analysis
showing that the original network inherits this good generalization bound from
its distillation, assuming the use of well-behaved data augmentation. This
bound is presented both in an abstract and in a concrete form, the latter
complemented by a reduction technique to handle modern computation graphs
featuring convolutional layers, fully-connected layers, and skip connections,
to name a few. To round out the story, a (looser) classical uniform convergence
analysis of compression is also presented, as well as a variety of experiments
on cifar and mnist demonstrating similar generalization performance between the
original network and its distillation.

该论文从理论上研究了高复杂度网络的蒸馏问题，并使用了数据增强来展示它的好的泛化性，并配合计算图缩减技术和实验来证明它具有实际应用价值。

通过蒸馏法获得推广界限

Generalization bounds via distillation

Metric learning networks are used to compute image embeddings, which are
widely used in many applications such as image retrieval and face recognition.
In this paper, we propose to use network distillation to efficiently compute
image embeddings with small networks. Network distillation has been
successfully applied to improve image classification, but has hardly been
explored for metric learning. To do so, we propose two new loss functions that
model the communication of a deep teacher network to a small student network.
We evaluate our system in several datasets, including CUB-200-2011, Cars-196,
Stanford Online Products and show that embeddings computed using small student
networks perform significantly better than those computed using standard
networks of similar size. Results on a very compact network (MobileNet-0.25),
which can be used on mobile devices, show that the proposed method can greatly
improve Recall@1 results from 27.5\% to 44.6\%. Furthermore, we investigate
various aspects of distillation for embeddings, including hint and attention
layers, semi-supervised learning and cross quality distillation. (Code is
available at this https URL)

本文提出使用网络蒸馏来高效计算小型网络产生的图像嵌入，通过构建两个新的损失函数模拟深度教师网络向小型学生网络通信的机制，在多个数据集上验证我们的方法，得出小型学生网络计算的嵌入明显优于使用相似大小的标准网络计算的结果。其中在可移动设备上使用的 MobileNet-0.25 产生了 44.6% 的 Recall@1 结果。同时，研究了蒸馏嵌入的各种方面，包括提示和注意力层，半监督学习以及不同质量的蒸馏。