Real-world data generation often involves certain geometries (e.g., graphs)
that induce instance-level interdependence. This characteristic makes the
generalization of learning models more difficult due to the intricate
interdependent patterns that impact data-generative distributions and can vary
from training to testing. In this work, we propose a geometric diffusion model
with learnable divergence fields for the challenging generalization problem
with interdependent data. We generalize the diffusion equation with stochastic
diffusivity at each time step, which aims to capture the multi-faceted
information flows among interdependent data. Furthermore, we derive a new
learning objective through causal inference, which can guide the model to learn
generalizable patterns of interdependence that are insensitive across domains.
Regarding practical implementation, we introduce three model instantiations
that can be considered as the generalized versions of GCN, GAT, and
Transformers, respectively, which possess advanced robustness against
distribution shifts. We demonstrate their promising efficacy for
out-of-distribution generalization on diverse real-world datasets.

我们提出了一个可学习的发散场几何扩散模型来解决依赖关系数据的挑战性泛化问题。通过引入随机扩散性来广义扩散方程，旨在捕捉依赖关系数据之间的多方面信息流动。另外，通过因果推断导出了一种新的学习目标，可指导模型学习对不同领域中都无关的依赖关系模式。在实际实现方面，我们提出了三种模型实例化方法，分别被视为 GCN、GAT 和 Transformers 的广义版本，具有对分布偏移的高鲁棒性。我们在不同的真实数据集上展示了这些模型在超出分布的泛化上的有希望的有效性。

学习发散场以获得偏移鲁棒的图表示

Learning Divergence Fields for Shift-Robust Graph Representations

Pac-Bayes bounds are among the most accurate generalization bounds for
classifiers learned from independently and identically distributed (IID) data,
and it is particularly so for margin classifiers: there have been recent
contributions showing how practical these bounds can be either to perform model
selection (Ambroladze et al., 2007) or even to directly guide the learning of
linear classifiers (Germain et al., 2009). However, there are many practical
situations where the training data show some dependencies and where the
traditional IID assumption does not hold. Stating generalization bounds for
such frameworks is therefore of the utmost interest, both from theoretical and
practical standpoints. In this work, we propose the first - to the best of our
knowledge - Pac-Bayes generalization bounds for classifiers trained on data
exhibiting interdependencies. The approach undertaken to establish our results
is based on the decomposition of a so-called dependency graph that encodes the
dependencies within the data, in sets of independent data, thanks to graph
fractional covers. Our bounds are very general, since being able to find an
upper bound on the fractional chromatic number of the dependency graph is
sufficient to get new Pac-Bayes bounds for specific settings. We show how our
results can be used to derive bounds for ranking statistics (such as Auc) and
classifiers trained on data distributed according to a stationary {\ss}-mixing
process. In the way, we show how our approach seemlessly allows us to deal with
U-processes. As a side note, we also provide a Pac-Bayes generalization bound
for classifiers learned on data from stationary $\varphi$-mixing distributions.

提出了第一篇关于在具有内在依赖性的数据上训练分类器的 Pac-Bayes 泛化界的研究，该方法基于依赖图的分解，通过图分数覆盖将依赖关系编码在数据的独立数据集中。