A curriculum is a planned sequence of learning materials and an effective one
can make learning efficient and effective for both humans and machines. Recent
studies developed effective data-driven curriculum learning approaches for
training graph neural networks in language applications. However, existing
curriculum learning approaches often employ a single criterion of difficulty in
their training paradigms. In this paper, we propose a new perspective on
curriculum learning by introducing a novel approach that builds on graph
complexity formalisms (as difficulty criteria) and model competence during
training. The model consists of a scheduling scheme which derives effective
curricula by accounting for different views of sample difficulty and model
competence during training. The proposed solution advances existing research in
curriculum learning for graph neural networks with the ability to incorporate a
fine-grained spectrum of graph difficulty criteria in their training paradigms.
Experimental results on real-world link prediction and node classification
tasks illustrate the effectiveness of the proposed approach.

通过引入一种建立在图复杂性规范和模型能力的新方法，本文在课程学习中提出了一种新的视角，该方法通过考虑不同样本困难度和模型能力的观点，在训练过程中推导出有效的课程表，为图神经网络的课程学习研究提供了进一步的发展。实验结果表明了该方法在实际的链接预测和节点分类任务中的有效性。

基于多视角、基于能力的图神经网络课程学习

Curriculum Learning for Graph Neural Networks: A Multiview  Competence-based Approach

Scheduled sampling is an effective method to alleviate the exposure bias
problem of neural machine translation. It simulates the inference scene by
randomly replacing ground-truth target input tokens with predicted ones during
training. Despite its success, its critical schedule strategies are merely
based on training steps, ignoring the real-time model competence, which limits
its potential performance and convergence speed. To address this issue, we
propose confidence-aware scheduled sampling. Specifically, we quantify
real-time model competence by the confidence of model predictions, based on
which we design fine-grained schedule strategies. In this way, the model is
exactly exposed to predicted tokens for high-confidence positions and still
ground-truth tokens for low-confidence positions. Moreover, we observe vanilla
scheduled sampling suffers from degenerating into the original teacher forcing
mode since most predicted tokens are the same as ground-truth tokens.
Therefore, under the above confidence-aware strategy, we further expose more
noisy tokens (e.g., wordy and incorrect word order) instead of predicted ones
for high-confidence token positions. We evaluate our approach on the
Transformer and conduct experiments on large-scale WMT 2014 English-German, WMT
2014 English-French, and WMT 2019 Chinese-English. Results show that our
approach significantly outperforms the Transformer and vanilla scheduled
sampling on both translation quality and convergence speed.

本文提出了基于置信度的定时采样策略，针对神经机器翻译中定时采样策略无法根据实时模型表现进行调整的问题，通过使用模型预测的置信度来量化实时模型表现，并设计了细粒度的采样策略。实验结果表明，该策略在 Transformer 模型上显著优于 Vanilla 定时采样，可以提高翻译质量和收敛速度。