Meta-learning is a powerful approach that exploits historical data to quickly
solve new tasks from the same distribution. In the low-data regime, methods
based on the closed-form posterior of Gaussian processes (GP) together with
Bayesian optimization have achieved high performance. However, these methods
are either computationally expensive or introduce assumptions that hinder a
principled propagation of uncertainty between task models. This may disrupt the
balance between exploration and exploitation during optimization. In this
paper, we develop ScaML-GP, a modular GP model for meta-learning that is
scalable in the number of tasks. Our core contribution is a carefully designed
multi-task kernel that enables hierarchical training and task scalability.
Conditioning ScaML-GP on the meta-data exposes its modular nature yielding a
test-task prior that combines the posteriors of meta-task GPs. In synthetic and
real-world meta-learning experiments, we demonstrate that ScaML-GP can learn
efficiently both with few and many meta-tasks.

在这篇论文中，我们开发了一种可扩展的基于高斯过程的模块化元学习模型 ScaML-GP，其中的核心贡献是一个经过精心设计的多任务核函数，它实现了层次化训练和任务的可扩展性。通过在元数据上对 ScaML-GP 进行条件化，我们揭示了其模块化特性，得到一个结合了元任务高斯过程后验的测试任务先验。在合成和真实世界的元学习实验中，我们证明了 ScaML-GP 可以在少量和大量元任务中高效学习。

高效可拓展的高斯过程元学习

Scalable Meta-Learning with Gaussian Processes

Catastrophic forgetting is a critical challenge in training deep neural
networks. Although continual learning has been investigated as a countermeasure
to the problem, it often suffers from the requirements of additional network
components and the limited scalability to a large number of tasks. We propose a
novel approach to continual learning by approximating a true loss function
using an asymmetric quadratic function with one of its sides overestimated. Our
algorithm is motivated by the empirical observation that the network parameter
updates affect the target loss functions asymmetrically. In the proposed
continual learning framework, we estimate an asymmetric loss function for the
tasks considered in the past through a proper overestimation of its unobserved
sides in training new tasks, while deriving the accurate model parameter for
the observable sides. In contrast to existing approaches, our method is free
from the side effects and achieves the state-of-the-art accuracy that is even
close to the upper-bound performance on several challenging benchmark datasets.

本文提出了一种利用非对称二次函数逼近真实损失函数的方法来解决深度神经网络中的灾难性遗忘问题和大规模任务可扩展性问题的连续学习框架，并通过多个挑战性基准数据集的实验结果表明该方法优于现有方法，且可达到接近上限性能。