Training nonlinear parametrizations such as deep neural networks to
numerically approximate solutions of partial differential equations is often
based on minimizing a loss that includes the residual, which is analytically
available in limited settings only. At the same time, empirically estimating
the training loss is challenging because residuals and related quantities can
have high variance, especially for transport-dominated and high-dimensional
problems that exhibit local features such as waves and coherent structures.
Thus, estimators based on data samples from un-informed, uniform distributions
are inefficient. This work introduces Neural Galerkin schemes that estimate the
training loss with data from adaptive distributions, which are empirically
represented via ensembles of particles. The ensembles are actively adapted by
evolving the particles with dynamics coupled to the nonlinear parametrizations
of the solution fields so that the ensembles remain informative for estimating
the training loss. Numerical experiments indicate that few dynamic particles
are sufficient for obtaining accurate empirical estimates of the training loss,
even for problems with local features and with high-dimensional spatial
domains.

通过使用 Neural Galerkin schemes 方法，利用集合较少的粒子动态适应粒子来近似求解偏微分方程的深度神经网络训练过程中的误差，并在具有局部特征和高维空间域的问题中获得了准确的实证误差估计。

神经 Galerkin 方案中的自适应采样：耦合参数和粒子动力学

Coupling parameter and particle dynamics for adaptive sampling in Neural  Galerkin schemes

This paper presents a unified approach based on Wasserstein distance to
derive concentration bounds for empirical estimates for two broad classes of
risk measures defined in the paper. The classes of risk measures introduced
include as special cases well known risk measures from the finance literature
such as conditional value at risk (CVaR), optimized certainty equivalent risk,
spectral risk measures, utility-based shortfall risk, cumulative prospect
theory (CPT) value, rank dependent expected utility and distorted risk
measures. Two estimation schemes are considered, one for each class of risk
measures. One estimation scheme involves applying the risk measure to the
empirical distribution function formed from a collection of i.i.d. samples of
the random variable (r.v.), while the second scheme involves applying the same
procedure to a truncated sample. The bounds provided apply to three popular
classes of distributions, namely sub-Gaussian, sub-exponential and heavy-tailed
distributions. The bounds are derived by first relating the estimation error to
the Wasserstein distance between the true and empirical distributions, and then
using recent concentration bounds for the latter. Previous concentration bounds
are available only for specific risk measures such as CVaR and CPT-value. The
bounds derived in this paper are shown to either match or improve upon previous
bounds in cases where they are available. The usefulness of the bounds is
illustrated through an algorithm and the corresponding regret bound for a
stochastic bandit problem involving a general risk measure from each of the two
classes introduced in the paper.

本文介绍了一种基于 Wasserstein 距离的统一方法，用于两类广泛定义的风险度量的经验估计的集中界限，其中提出的风险度量类包括 CVaR、谱风险度量、CPT 值、偏差风险度量等。