Continual learning (CL) aims to train deep neural networks efficiently on
streaming data while limiting the forgetting caused by new tasks. However,
learning transferable knowledge with less interference between tasks is
difficult, and real-world deployment of CL models is limited by their inability
to measure predictive uncertainties. To address these issues, we propose
handling CL tasks with neural processes (NPs), a class of meta-learners that
encode different tasks into probabilistic distributions over functions all
while providing reliable uncertainty estimates. Specifically, we propose an
NP-based CL approach (NPCL) with task-specific modules arranged in a
hierarchical latent variable model. We tailor regularizers on the learned
latent distributions to alleviate forgetting. The uncertainty estimation
capabilities of the NPCL can also be used to handle the task head/module
inference challenge in CL. Our experiments show that the NPCL outperforms
previous CL approaches. We validate the effectiveness of uncertainty estimation
in the NPCL for identifying novel data and evaluating instance-level model
confidence. Code is available at https://github.com/srvCodes/NPCL.

使用神经过程处理连续学习任务，通过任务特定模块和层次潜在变量模型，提供可靠的不确定性估计，并在不同任务间减轻遗忘现象。

NPCL：针对不确定性感知的连续学习的神经过程

NPCL: Neural Processes for Uncertainty-Aware Continual Learning

Translation into morphologically-rich languages challenges neural machine
translation (NMT) models with extremely sparse vocabularies where atomic
treatment of surface forms is unrealistic. This problem is typically addressed
by either pre-processing words into subword units or performing translation
directly at the level of characters. The former is based on word segmentation
algorithms optimized using corpus-level statistics with no regard to the
translation task. The latter learns directly from translation data but requires
rather deep architectures. In this paper, we propose to translate words by
modeling word formation through a hierarchical latent variable model which
mimics the process of morphological inflection. Our model generates words one
character at a time by composing two latent representations: a continuous one,
aimed at capturing the lexical semantics, and a set of (approximately) discrete
features, aimed at capturing the morphosyntactic function, which are shared
among different surface forms. Our model achieves better accuracy in
translation into three morphologically-rich languages than conventional
open-vocabulary NMT methods, while also demonstrating a better generalization
capacity under low to mid-resource settings.

本文提出了一种通过分层潜在变量模拟词态变化过程的方法，通过组合两个潜在表示（一个连续的表示和一组（近似）离散特征），逐个字符生成单词，从而使神经机器翻译在三种形态丰富的语言中的精度和资源利用率得到了提高。