Gradient-based meta-learning techniques aim to distill useful prior knowledge from a set of training tasks such that new tasks can be learned more efficiently with gradient descent. While these methods have achieved successes in various scenarios, they commonly adapt all parameters of trainable layers when learning new tasks. This neglects potentially more efficient learning strategies for a given task distribution and may be susceptible to overfitting, especially in few-shot learning where tasks must be learned from a limited number of examples. To address these issues, we propose Subspace Adaptation Prior (SAP), a novel gradient-based meta-learning algorithm that jointly learns good initialization parameters (prior knowledge) and layer-wise parameter subspaces in the form of operation subsets that should be adaptable. In this way, SAP can learn which operation subsets to adjust with gradient descent based on the underlying task distribution, simultaneously decreasing the risk of overfitting when learning new tasks. We demonstrate that this ability is helpful as SAP yields superior or competitive performance in few-shot image classification settings (gains between 0.1% and 3.9% in accuracy). Analysis of the learned subspaces demonstrates that low-dimensional operations often yield high activation strengths, indicating that they may be important for achieving good few-shot learning performance. For reproducibility purposes, we publish all our research code publicly.

梯度基于元学习技术旨在从一组训练任务中提取有用的先验知识，以便可以更高效地使用梯度下降学习新任务。我们提出了基于子空间适应先验（SAP）的梯度元学习算法，它不仅可以学习好的初始化参数（先验知识），还可以学习可调整的层参数子空间，以减少过拟合的风险并在少样本学习中获得优越的性能。数值实验表明，SAP在少样本图像分类设置中表现出优越或具有竞争力的性能。

少样本学习的子空间适应先验