McKean-Vlasov stochastic differential equations (MV-SDEs) provide a
mathematical description of the behavior of an infinite number of interacting
particles by imposing a dependence on the particle density. As such, we study
the influence of explicitly including distributional information in the
parameterization of the SDE. We propose a series of semi-parametric methods for
representing MV-SDEs, and corresponding estimators for inferring parameters
from data based on the properties of the MV-SDE. We analyze the characteristics
of the different architectures and estimators, and consider their applicability
in relevant machine learning problems. We empirically compare the performance
of the different architectures and estimators on real and synthetic datasets
for time series and probabilistic modeling. The results suggest that explicitly
including distributional dependence in the parameterization of the SDE is
effective in modeling temporal data with interaction under an exchangeability
assumption while maintaining strong performance for standard It\^o-SDEs due to
the richer class of probability flows associated with MV-SDEs.

将分布依赖显式地包含在随机微分方程的参数化中对于建模具有交互性的时间数据是有效的，并在维持标准 Itô-SDE 的优越性能的同时，通过 MV-SDE 关联的更丰富的概率流类来进行时间数据的建模。

神经麦克林 - 瓦萨夫过程：扩散过程中的分布依赖

Neural McKean-Vlasov Processes: Distributional Dependence in Diffusion  Processes

It has been shown that semi-parametric methods, which combine standard neural
networks with non-parametric components such as external memory modules and
data retrieval, are particularly helpful in data scarcity and
out-of-distribution (OOD) scenarios. However, existing semi-parametric methods
mostly depend on independent raw data points - this strategy is difficult to
scale up due to both high computational costs and the incapacity of current
attention mechanisms with a large number of tokens. In this paper, we introduce
a novel heterogeneous memory augmentation approach for neural networks which,
by introducing learnable memory tokens with attention mechanism, can
effectively boost performance without huge computational overhead. Our
general-purpose method can be seamlessly combined with various backbones (MLP,
CNN, GNN, and Transformer) in a plug-and-play manner. We extensively evaluate
our approach on various image and graph-based tasks under both in-distribution
(ID) and OOD conditions and show its competitive performance against
task-specific state-of-the-art methods. Code is available at
https://github.com/qiuzh20/HMA.

通过引入可学习的记忆令牌和注意机制，我们提出了一种新颖的异构记忆增强方法，可以有效提升性能而不需要巨大的计算开销。

异构存储增强神经网络

Heterogenous Memory Augmented Neural Networks

Single-index models are a class of functions given by an unknown univariate
``link'' function applied to an unknown one-dimensional projection of the
input. These models are particularly relevant in high dimension, when the data
might present low-dimensional structure that learning algorithms should adapt
to. While several statistical aspects of this model, such as the sample
complexity of recovering the relevant (one-dimensional) subspace, are
well-understood, they rely on tailored algorithms that exploit the specific
structure of the target function. In this work, we introduce a natural class of
shallow neural networks and study its ability to learn single-index models via
gradient flow. More precisely, we consider shallow networks in which biases of
the neurons are frozen at random initialization. We show that the corresponding
optimization landscape is benign, which in turn leads to generalization
guarantees that match the near-optimal sample complexity of dedicated
semi-parametric methods.

本文研究了一种自然的浅层神经网络，通过梯度流学习单指数模型，表明这种方法的优化效果良好，并且具有与半参数方法相当的泛化能力。