A wide range of graph embedding objectives decompose into two components: one
that attracts the embeddings of nodes that are perceived as similar, and
another that repels embeddings of nodes that are perceived as dissimilar.
Because real-world graphs are sparse and the number of dissimilar pairs grows
quadratically with the number of nodes, Skip-Gram Negative Sampling (SGNS) has
emerged as a popular and efficient repulsion approach. SGNS repels each node
from a sample of dissimilar nodes, as opposed to all dissimilar nodes. In this
work, we show that node-wise repulsion is, in aggregate, an approximate
re-centering of the node embedding dimensions. Such dimension operations are
much more scalable than node operations. The dimension approach, in addition to
being more efficient, yields a simpler geometric interpretation of the
repulsion. Our result extends findings from the self-supervised learning
literature to the skip-gram model, establishing a connection between skip-gram
node contrast and dimension regularization. We show that in the limit of large
graphs, under mild regularity conditions, the original node repulsion objective
converges to optimization with dimension regularization. We use this
observation to propose an algorithm augmentation framework that speeds up any
existing algorithm, supervised or unsupervised, using SGNS. The framework
prioritizes node attraction and replaces SGNS with dimension regularization. We
instantiate this generic framework for LINE and node2vec and show that the
augmented algorithms preserve downstream performance while dramatically
increasing efficiency.

将节点吸引和维度规范化作为替代方法的图嵌入优化技术，从而提高算法效率。

重新审视 Skip-Gram 负采样：在图嵌入中进行尺寸正则化以实现更高效的差异性保留

Re-visiting Skip-Gram Negative Sampling: Dimension Regularization for  More Efficient Dissimilarity Preservation in Graph Embeddings

Skip-Gram Negative Sampling (SGNS) word embedding model, well known by its
implementation in "word2vec" software, is usually optimized by stochastic
gradient descent. However, the optimization of SGNS objective can be viewed as
a problem of searching for a good matrix with the low-rank constraint. The most
standard way to solve this type of problems is to apply Riemannian optimization
framework to optimize the SGNS objective over the manifold of required low-rank
matrices. In this paper, we propose an algorithm that optimizes SGNS objective
using Riemannian optimization and demonstrates its superiority over popular
competitors, such as the original method to train SGNS and SVD over SPPMI
matrix.

本文提出一种基于黎曼优化的算法，有效优化 Skip-Gram Negative Sampling（SGNS）词嵌入模型，与传统方法和基于 SPPMI 矩阵的 SVD 相比表现更优。