Gradient based meta-learning methods are prone to overfit on the
meta-training set, and this behaviour is more prominent with large and complex
networks. Moreover, large networks restrict the application of meta-learning
models on low-power edge devices. While choosing smaller networks avoid these
issues to a certain extent, it affects the overall generalization leading to
reduced performance. Clearly, there is an approximately optimal choice of
network architecture that is best suited for every meta-learning problem,
however, identifying it beforehand is not straightforward. In this paper, we
present MetaDOCK, a task-specific dynamic kernel selection strategy for
designing compressed CNN models that generalize well on unseen tasks in
meta-learning. Our method is based on the hypothesis that for a given set of
similar tasks, not all kernels of the network are needed by each individual
task. Rather, each task uses only a fraction of the kernels, and the selection
of the kernels per task can be learnt dynamically as a part of the inner update
steps. MetaDOCK compresses the meta-model as well as the task-specific inner
models, thus providing significant reduction in model size for each task, and
through constraining the number of active kernels for every task, it implicitly
mitigates the issue of meta-overfitting. We show that for the same inference
budget, pruned versions of large CNN models obtained using our approach
consistently outperform the conventional choices of CNN models. MetaDOCK
couples well with popular meta-learning approaches such as iMAML. The efficacy
of our method is validated on CIFAR-fs and mini-ImageNet datasets, and we have
observed that our approach can provide improvements in model accuracy of up to
2% on standard meta-learning benchmark, while reducing the model size by more
than 75%.

本文提出了一种名为 MetaDOCK 的任务特定动态内核选择策略，通过良好地压缩 CNN 和任务特定的内部模型，可以在减少模型大小的同时提高模型的准确性。