Reinforcement learning (RL) is an effective method of finding reasoning
pathways in incomplete knowledge graphs (KGs). To overcome the challenges of a
large action space, a self-supervised pre-training method is proposed to warm
up the policy network before the RL training stage. To alleviate the
distributional mismatch issue in general self-supervised RL (SSRL), in our
supervised learning (SL) stage, the agent selects actions based on the policy
network and learns from generated labels; this self-generation of labels is the
intuition behind the name self-supervised. With this training framework, the
information density of our SL objective is increased and the agent is prevented
from getting stuck with the early rewarded paths. Our self-supervised RL (SSRL)
method improves the performance of RL by pairing it with the wide coverage
achieved by SL during pretraining, since the breadth of the SL objective makes
it infeasible to train an agent with that alone. We show that our SSRL model
meets or exceeds current state-of-the-art results on all Hits@k and mean
reciprocal rank (MRR) metrics on four large benchmark KG datasets. This SSRL
method can be used as a plug-in for any RL architecture for a KGR task. We
adopt two RL architectures, i.e., MINERVA and MultiHopKG as our baseline RL
models and experimentally show that our SSRL model consistently outperforms
both baselines on all of these four KG reasoning tasks. Full code for the paper
available at
this https URL

强化学习通过自我监督预训练方法改进了知识图谱推理任务的性能，解决了动作空间大的挑战和分布不匹配问题，并且超过了现有的最先进结果。

知识图谱推理：自我监督强化学习

Knowledge Graph Reasoning with Self-supervised Reinforcement Learning

The ability to compare the semantic similarity between text corpora is
important in a variety of natural language processing applications. However,
standard methods for evaluating these metrics have yet to be established. We
propose a set of automatic and interpretable measures for assessing the
characteristics of corpus-level semantic similarity metrics, allowing sensible
comparison of their behavior. We demonstrate the effectiveness of our
evaluation measures in capturing fundamental characteristics by evaluating them
on a collection of classical and state-of-the-art metrics. Our measures
revealed that recently-developed metrics are becoming better in identifying
semantic distributional mismatch while classical metrics are more sensitive to
perturbations in the surface text levels.

通过提出一系列的评估度量，本文对语义相似度度量的特征进行自动和可解释的评估，从而实现了对不同语义相似度度量方法行为的合理比较。通过对经典方法和最新方法的评估，我们的度量揭示了最近开发的度量在识别语义分布不匹配方面变得更好，而经典度量则对表面文本水平的扰动更加敏感。