The ability to compose learned skills to solve new tasks is an important
property of lifelong-learning agents. In this work, we formalise the logical
composition of tasks as a Boolean algebra. This allows us to formulate new
tasks in terms of the negation, disjunction and conjunction of a set of base
tasks. We then show that by learning goal-oriented value functions and
restricting the transition dynamics of the tasks, an agent can solve these new
tasks with no further learning. We prove that by composing these value
functions in specific ways, we immediately recover the optimal policies for all
tasks expressible under the Boolean algebra. We verify our approach in two
domains---including a high-dimensional video game environment requiring
function approximation---where an agent first learns a set of base skills, and
then composes them to solve a super-exponential number of new tasks.

本文提出了一种以布尔代数为基础的逻辑任务组合形式，通过学习目标导向的价值函数和约束任务的转换动态，以指定的方式组合这些价值函数后，可以立即恢复布尔代数下所有表达式的最优策略，并在高维度视频游戏环境和其他领域中进行验证。

强化学习布尔任务代数

A Boolean Task Algebra for Reinforcement Learning

The world is fundamentally compositional, so it is natural to think of visual
recognition as the recognition of basic visually primitives that are composed
according to well-defined rules. This strategy allows us to recognize unseen
complex concepts from simple visual primitives. However, the current trend in
visual recognition follows a data greedy approach where huge amounts of data
are required to learn models for any desired visual concept. In this paper, we
build on the compositionality principle and develop an "algebra" to compose
classifiers for complex visual concepts. To this end, we learn neural network
modules to perform boolean algebra operations on simple visual classifiers.
Since these modules form a complete functional set, a classifier for any
complex visual concept defined as a boolean expression of primitives can be
obtained by recursively applying the learned modules, even if we do not have a
single training sample. As our experiments show, using such a framework, we can
compose classifiers for complex visual concepts outperforming standard
baselines on two well-known visual recognition benchmarks. Finally, we present
a qualitative analysis of our method and its properties.

本文提出了一种使用神经网络模块执行布尔代数运算来构建分类器的方法，通过递归应用这些模块，即使没有单个训练样本，也可以获得任何由基本元素的布尔表达式定义的复杂视觉概念的分类器，实验证明，使用这种框架可以组合复杂的视觉概念分类器，并在两个知名的视觉识别基准测试中优于标准基准线。