We propose a novel hierarchical Bayesian model for learning with a large
(possibly infinite) number of tasks/episodes, which suits well the few-shot
meta learning problem. We consider episode-wise random variables to model
episode-specific target generative processes, where these local random
variables are governed by a higher-level global random variate. The global
variable helps memorize the important information from historic episodes while
controlling how much the model needs to be adapted to new episodes in a
principled Bayesian manner. Within our model framework, the prediction on a
novel episode/task can be seen as a Bayesian inference problem. However, a main
obstacle in learning with a large/infinite number of local random variables in
online nature, is that one is not allowed to store the posterior distribution
of the current local random variable for frequent future updates, typical in
conventional variational inference. We need to be able to treat each local
variable as a one-time iterate in the optimization. We propose a
Normal-Inverse-Wishart model, for which we show that this one-time iterate
optimization becomes feasible due to the approximate closed-form solutions for
the local posterior distributions. The resulting algorithm is more attractive
than the MAML in that it is not required to maintain computational graphs for
the whole gradient optimization steps per episode. Our approach is also
different from existing Bayesian meta learning methods in that unlike dealing
with a single random variable for the whole episodes, our approach has a
hierarchical structure that allows one-time episodic optimization, desirable
for principled Bayesian learning with many/infinite tasks. The code is
available at https://github.com/minyoungkim21/niwmeta.

我们提出了一种新的层次贝叶斯模型，可用于针对大量（可能是无限）任务 / 情节进行学习，适合 few-shot 元学习问题。