Modular approaches, which use a different composition of modules for each
problem and avoid forgetting by design, have been shown to be a promising
direction in continual learning (CL). However, searching through the large,
discrete space of possible module compositions is a challenge because
evaluating a composition's performance requires a round of neural network
training. To address this challenge, we develop a modular CL framework, called
PICLE, that accelerates search by using a probabilistic model to cheaply
compute the fitness of each composition. The model combines prior knowledge
about good module compositions with dataset-specific information. Its use is
complemented by splitting up the search space into subsets, such as perceptual
and latent subsets. We show that PICLE is the first modular CL algorithm to
achieve different types of transfer while scaling to large search spaces. We
evaluate it on two benchmark suites designed to capture different desiderata of
CL techniques. On these benchmarks, PICLE offers significantly better
performance than state-of-the-art CL baselines.

PICLE 是一种模块化的持续学习（CL）框架，使用概率建模加速搜索，在不同类型的转移中较好地表现，应用于两个基准测试套件表现优异。

基于概率的模块化持续学习框架

A Probabilistic Framework for Modular Continual Learning

In order to mitigate some of the inefficiencies of Reinforcement Learning
(RL), modular approaches composing different decision-making policies to derive
agents capable of performing a variety of tasks have been proposed. The modules
at the basis of these architectures are generally reusable, also allowing for
"plug-and-play" integration. However, such solutions still lack the ability to
process and integrate multiple types of information (knowledge), such as rules,
sub-goals, and skills. We propose Augmented Modular Reinforcement Learning
(AMRL) to address these limitations. This new framework uses an arbitrator to
select heterogeneous modules and seamlessly incorporate different types of
knowledge. Additionally, we introduce a variation of the selection mechanism,
namely the Memory-Augmented Arbitrator, which adds the capability of exploiting
temporal information. We evaluate the proposed mechanisms on established as
well as new environments and benchmark them against prominent deep RL
algorithms. Our results demonstrate the performance improvements that can be
achieved by augmenting traditional modular RL with other forms of heterogeneous
knowledge.

通过使用调解者选择异质性模块和平稳地融合不同类型的知识，提出了增强型模块化强化学习 (AMRL) 框架，并引入了选择机制的变体，即记忆增强型调解者，以利用时间信息，评估结果表明增强传统模块化 RL 的其他形式的异质知识可以提高性能

基于异构知识的增强模块化强化学习

Augmented Modular Reinforcement Learning based on Heterogeneous  Knowledge

In modern applications, statisticians are faced with integrating
heterogeneous data modalities relevant for an inference, prediction, or
decision problem. In such circumstances, it is convenient to use a graphical
model to represent the statistical dependencies, via a set of connected
"modules", each relating to a specific data modality, and drawing on specific
domain expertise in their development. In principle, given data, the
conventional statistical update then allows for coherent uncertainty
quantification and information propagation through and across the modules.
However, misspecification of any module can contaminate the estimate and update
of others, often in unpredictable ways. In various settings, particularly when
certain modules are trusted more than others, practitioners have preferred to
avoid learning with the full model in favor of approaches that restrict the
information propagation between modules, for example by restricting propagation
to only particular directions along the edges of the graph. In this article, we
investigate why these modular approaches might be preferable to the full model
in misspecified settings. We propose principled criteria to choose between
modular and full-model approaches. The question arises in many applied
settings, including large stochastic dynamical systems, meta-analysis,
epidemiological models, air pollution models,
pharmacokinetics-pharmacodynamics, and causal inference with propensity scores.

本文研究为何在误定义环节下，模块化方法比完整模型更可取，并提出选择模块化和完整模型方法的原则标准。