Machine learning-based performance models are increasingly being used to
build critical job scheduling and application optimization decisions.
Traditionally, these models assume that data distribution does not change as
more samples are collected over time. However, owing to the complexity and
heterogeneity of production HPC systems, they are susceptible to hardware
degradation, replacement, and/or software patches, which can lead to drift in
the data distribution that can adversely affect the performance models. To this
end, we develop continually learning performance models that account for the
distribution drift, alleviate catastrophic forgetting, and improve
generalizability. Our best model was able to retain accuracy, regardless of
having to learn the new distribution of data inflicted by system changes, while
demonstrating a 2x improvement in the prediction accuracy of the whole data
sequence in comparison to the naive approach.

基于机器学习的性能模型在构建关键的作业调度和应用程序优化决策中越来越被使用。我们开发了持续学习性能模型，考虑到数据分布漂移，减轻灾难性遗忘，并提高泛化能力。我们的最佳模型能够保持准确性，无论系统变化引起的新数据分布，同时相比于朴素方法，在整个数据序列的预测准确度上提升了 2 倍。

朝向持续学习的应用性能模型

Towards Continually Learning Application Performance Models

Temporal domain generalization is a promising yet extremely challenging area
where the goal is to learn models under temporally changing data distributions
and generalize to unseen data distributions following the trends of the change.
The advancement of this area is challenged by: 1) characterizing data
distribution drift and its impacts on models, 2) expressiveness in tracking the
model dynamics, and 3) theoretical guarantee on the performance. To address
them, we propose a Temporal Domain Generalization with Drift-Aware Dynamic
Neural Network (DRAIN) framework. Specifically, we formulate the problem into a
Bayesian framework that jointly models the relation between data and model
dynamics. We then build a recurrent graph generation scenario to characterize
the dynamic graph-structured neural networks learned across different time
points. It captures the temporal drift of model parameters and data
distributions and can predict models in the future without the presence of
future data. In addition, we explore theoretical guarantees of the model
performance under the challenging temporal DG setting and provide theoretical
analysis, including uncertainty and generalization error. Finally, extensive
experiments on several real-world benchmarks with temporal drift demonstrate
the effectiveness and efficiency of the proposed method.

提出了一种名为 DRAIN 的方法来解决时域领域通用问题，通过 Bayesian 方法建立动态图结构神经网络，在考虑模型参数和数据分布在不同时间点的动态漂移的同时，提供了理论保证，具有较高的有效性和效率。

基于漂移感知动态神经网络的时域泛化

Temporal Domain Generalization with Drift-Aware Dynamic Neural Networks

Classical machine learning algorithms often assume that the data are drawn
i.i.d. from a stationary probability distribution. Recently, continual learning
emerged as a rapidly growing area of machine learning where this assumption is
relaxed, i.e. where the data distribution is non-stationary and changes over
time. This paper represents the state of data distribution by a context
variable $c$. A drift in $c$ leads to a data distribution drift.
A context drift may change the target distribution, the input distribution,
or both. Moreover, distribution drifts might be abrupt or gradual. In continual
learning, context drifts may interfere with the learning process and erase
previously learned knowledge; thus, continual learning algorithms must include
specialized mechanisms to deal with such drifts. In this paper, we aim to
identify and categorize different types of context drifts and potential
assumptions about them, to better characterize various continual-learning
scenarios. Moreover, we propose to use the distribution drift framework to
provide more precise definitions of several terms commonly used in the
continual learning field.

本文讨论了机器学习中的持续学习问题，通过引入上下文变量来解决数据分布的变动所引起的问题，并提出了处理上下文变动的机制以及通过数据分布漂移框架提供持续学习领域中常用术语的更精确定义。