We focus on the fundamental mathematical structure of score-based generative
models (SGMs). We first formulate SGMs in terms of the Wasserstein proximal
operator (WPO) and demonstrate that, via mean-field games (MFGs), the WPO
formulation reveals mathematical structure that describes the inductive bias of
diffusion and score-based models. In particular, MFGs yield optimality
conditions in the form of a pair of coupled partial differential equations: a
forward-controlled Fokker-Planck (FP) equation, and a backward
Hamilton-Jacobi-Bellman (HJB) equation. Via a Cole-Hopf transformation and
taking advantage of the fact that the cross-entropy can be related to a linear
functional of the density, we show that the HJB equation is an uncontrolled FP
equation. Second, with the mathematical structure at hand, we present an
interpretable kernel-based model for the score function which dramatically
improves the performance of SGMs in terms of training samples and training
time. In addition, the WPO-informed kernel model is explicitly constructed to
avoid the recently studied memorization effects of score-based generative
models. The mathematical form of the new kernel-based models in combination
with the use of the terminal condition of the MFG reveals new explanations for
the manifold learning and generalization properties of SGMs, and provides a
resolution to their memorization effects. Finally, our mathematically informed,
interpretable kernel-based model suggests new scalable bespoke neural network
architectures for high-dimensional applications.

我们着眼于得分基础生成模型（SGMs）的基本数学结构，以 Wasserstein 邻近算子（WPO）为基础进行了 SGMs 的数学形式重构，并通过均匀场博弈（MFGs）揭示出描述扩散和基于得分模型的归纳偏差的数学结构。借助 Cole-Hopf 变换和交叉熵与密度的线性功能之间的关系，我们发现 HJB 方程是一个无控制的 FP 方程。其次，基于得到的数学结构，我们提出了一个可解释的基于核函数的得分函数模型，大大提升了 SGMs 在训练样本和训练时间方面的性能。新的基于核函数的模型的数学形式与 MFG 的终端条件的使用揭示了 SGMs 的流形学习和泛化特性的新解释，并解决了它们的记忆效应问题。最后，我们提出了一种基于数学信息和可解释性的基于核函数的模型，为高维应用提供了新的可扩展的定制神经网络架构。

Wasserstein 近似算子描述基于评分的生成模型并解决记忆问题

Wasserstein proximal operators describe score-based generative models  and resolve memorization

With rising uncertainty in the real world, online Reinforcement Learning (RL)
has been receiving increasing attention due to its fast learning capability and
improving data efficiency. However, online RL often suffers from complex Value
Function Approximation (VFA) and catastrophic interference, creating difficulty
for the deep neural network to be applied to an online RL algorithm in a fully
online setting. Therefore, a simpler and more adaptive approach is introduced
to evaluate value function with the kernel-based model. Sparse representations
are superior at handling interference, indicating that competitive sparse
representations should be learnable, non-prior, non-truncated and explicit when
compared with current sparse representation methods. Moreover, in learning
sparse representations, attention mechanisms are utilized to represent the
degree of sparsification, and a smooth attentive function is introduced into
the kernel-based VFA. In this paper, we propose an Online Attentive
Kernel-Based Temporal Difference (OAKTD) algorithm using two-timescale
optimization and provide convergence analysis of our proposed algorithm.
Experimental evaluations showed that OAKTD outperformed several Online
Kernel-based Temporal Difference (OKTD) learning algorithms in addition to the
Temporal Difference (TD) learning algorithm with Tile Coding on public Mountain
Car, Acrobot, CartPole and Puddle World tasks.

本文提出了一种在线注意力核在线学习算法 (OAKTD)，使用核模型的基值函数来评估价值函数，其中利用到了稀疏表示和注意机制，通过实验评估发现 OAKTD 在一些公共任务上优于其他在线 Kernel-based Temporal Difference (OKTD) learning algorithms 以及使用 Tile Coding 的 Temporal Difference (TD) learning algorithm。