In regularization Self-Supervised Learning (SSL) methods for graphs,
computational complexity increases with the number of nodes in graphs and
embedding dimensions. To mitigate the scalability of non-contrastive graph SSL,
we propose a novel approach to reduce the cost of computing the covariance
matrix for the pre-training loss function with volume-maximization terms. Our
work focuses on reducing the cost associated with the loss computation via
graph node or dimension sampling. We provide theoretical insight into why
dimension sampling would result in accurate loss computations and support it
with mathematical derivation of the novel approach. We develop our experimental
setup on the node-level graph prediction tasks, where SSL pre-training has
shown to be difficult due to the large size of real world graphs. Our
experiments demonstrate that the cost associated with the loss computation can
be reduced via node or dimension sampling without lowering the downstream
performance. Our results demonstrate that sampling mostly results in improved
downstream performance. Ablation studies and experimental analysis are provided
to untangle the role of the different factors in the experimental setup.

通过采样节点或维度，可以降低损失计算成本，并且不会降低下游性能。