Deep Reinforcement Learning (deep RL) has made several breakthroughs in
recent years in applications ranging from complex control tasks in unmanned
vehicles to game playing. Despite their success, deep RL still lacks several
important capacities of human intelligence, such as transfer learning,
abstraction and interpretability. Deep Symbolic Reinforcement Learning (DSRL)
seeks to incorporate such capacities to deep Q-networks (DQN) by learning a
relevant symbolic representation prior to using Q-learning. In this paper, we
propose a novel extension of DSRL, which we call Symbolic Reinforcement
Learning with Common Sense (SRL+CS), offering a better balance between
generalization and specialization, inspired by principles of common sense when
assigning rewards and aggregating Q-values. Experiments reported in this paper
show that SRL+CS learns consistently faster than Q-learning and DSRL, achieving
also a higher accuracy. In the hardest case, where agents were trained in a
deterministic environment and tested in a random environment, SRL+CS achieves
nearly 100% average accuracy compared to DSRL's 70% and DQN's 50% accuracy. To
the best of our knowledge, this is the first case of near perfect zero-shot
transfer learning using Reinforcement Learning.

本论文提出了一种名为 Symbolic Reinforcement Learning with Common Sense (SRL+CS) 的算法，它在奖励分配和 Q 值聚合时结合了常识原则，使得在转移学习和零 - shot 转移学习等方面具有更好的泛化和特化表现。实验结果表明，SRL+CS 算法比 Q-learning 和 DSRL 算法更为快速且更准确，是近乎完美零 - shot 转移学习在强化学习领域的首次尝试。

走向具备常识的符号强化学习

Towards Symbolic Reinforcement Learning with Common Sense

Adequate representation of natural language semantics requires access to vast
amounts of common sense and domain-specific world knowledge. Prior work in the
field was based on purely statistical techniques that did not make use of
background knowledge, on limited lexicographic knowledge bases such as WordNet,
or on huge manual efforts such as the CYC project. Here we propose a novel
method, called Explicit Semantic Analysis (ESA), for fine-grained semantic
interpretation of unrestricted natural language texts. Our method represents
meaning in a high-dimensional space of concepts derived from Wikipedia, the
largest encyclopedia in existence. We explicitly represent the meaning of any
text in terms of Wikipedia-based concepts. We evaluate the effectiveness of our
method on text categorization and on computing the degree of semantic
relatedness between fragments of natural language text. Using ESA results in
significant improvements over the previous state of the art in both tasks.
Importantly, due to the use of natural concepts, the ESA model is easy to
explain to human users.

本文提出一种名为 Explicit Semantic Analysis（ESA）的新方法，基于 Wikipedia 的概念表示未受限制的自然语言文本的含义，用于文本分类和计算自然语言文本片段之间的语义相关性，此方法利用自然概念使 ESA 模型易于向人类用户解释，并在这两个任务中取得了显著的改进。