Representation Learning on Knowledge Graphs (KGs) is essential for downstream
tasks. The dominant approach, KG Embedding (KGE), represents entities with
independent vectors and faces the scalability challenge. Recent studies propose
an alternative way for parameter efficiency, which represents entities by
composing entity-corresponding codewords matched from predefined small-scale
codebooks. We refer to the process of obtaining corresponding codewords of each
entity as entity quantization, for which previous works have designed
complicated strategies. Surprisingly, this paper shows that simple random
entity quantization can achieve similar results to current strategies. We
analyze this phenomenon and reveal that entity codes, the quantization outcomes
for expressing entities, have higher entropy at the code level and Jaccard
distance at the codeword level under random entity quantization. Therefore,
different entities become more easily distinguished, facilitating effective KG
representation. The above results show that current quantization strategies are
not critical for KG representation, and there is still room for improvement in
entity distinguishability beyond current strategies. The code to reproduce our
results is available at this https URL

知识图谱上的表示学习对下游任务至关重要，本研究展示了简单随机实体量化方法能实现与当前策略相似的结果，表明当前量化策略并不关键，实体区分度仍有改善空间。

随机实体量化用于参数高效的组合知识图表示

Random Entity Quantization for Parameter-Efficient Compositional  Knowledge Graph Representation

A novel deep learning method for improving the belief propagation algorithm
is proposed. The method generalizes the standard belief propagation algorithm
by assigning weights to the edges of the Tanner graph. These edges are then
trained using deep learning techniques. A well-known property of the belief
propagation algorithm is the independence of the performance on the transmitted
codeword. A crucial property of our new method is that our decoder preserved
this property. Furthermore, this property allows us to learn only a single
codeword instead of exponential number of code-words. Improvements over the
belief propagation algorithm are demonstrated for various high density parity
check codes.

提出了一种基于深度学习方法改进置信传播算法的新方法。该方法通过对 Tanner 图的边进行加权来推广标准置信传播算法，然后使用深度学习技术进行训练。该方法能够保留传统算法的性能独立性，从而只需要学习一个码字而不是指数数量的码字，并在各种纠错码中展示了对置信传播算法的改进。