Deep equilibrium (DEQ) models are widely recognized as a memory efficient
alternative to standard neural networks, achieving state-of-the-art performance
in language modeling and computer vision tasks. These models solve a fixed
point equation instead of explicitly computing the output, which sets them
apart from standard neural networks. However, existing DEQ models often lack
formal guarantees of the existence and uniqueness of the fixed point, and the
convergence of the numerical scheme used for computing the fixed point is not
formally established. As a result, DEQ models are potentially unstable in
practice. To address these drawbacks, we introduce a novel class of DEQ models
called positive concave deep equilibrium (pcDEQ) models. Our approach, which is
based on nonlinear Perron-Frobenius theory, enforces nonnegative weights and
activation functions that are concave on the positive orthant. By imposing
these constraints, we can easily ensure the existence and uniqueness of the
fixed point without relying on additional complex assumptions commonly found in
the DEQ literature, such as those based on monotone operator theory in convex
analysis. Furthermore, the fixed point can be computed with the standard fixed
point algorithm, and we provide theoretical guarantees of geometric
convergence, which, in particular, simplifies the training process. Experiments
demonstrate the competitiveness of our pcDEQ models against other implicit
models.

通过引入一种名为正凹深度平衡 (pcDEQ) 模型的新型 DEQ 模型类，通过基于非线性 Perron-Frobenius 理论的方法强制非负权重和激活函数，可以简化训练过程并且保证了固定点的存在和唯一性，进而解决了 DEQ 模型存在的固定点唯一性和收敛性问题，实验证明了 pcDEQ 模型在隐式模型中的竞争力。

正凸深层均衡模型

Positive concave deep equilibrium models

Green's function characterizes a partial differential equation (PDE) and maps
its solution in the entire domain as integrals. Finding the analytical form of
Green's function is a non-trivial exercise, especially for a PDE defined on a
complex domain or a PDE with variable coefficients. In this paper, we propose a
novel boundary integral network to learn the domain-independent Green's
function, referred to as BIN-G. We evaluate the Green's function in the BIN-G
using a radial basis function (RBF) kernel-based neural network. We train the
BIN-G by minimizing the residual of the PDE and the mean squared errors of the
solutions to the boundary integral equations for prescribed test functions. By
leveraging the symmetry of the Green's function and controlling refinements of
the RBF kernel near the singularity of the Green function, we demonstrate that
our numerical scheme enables fast training and accurate evaluation of the
Green's function for PDEs with variable coefficients. The learned Green's
function is independent of the domain geometries, forcing terms, and boundary
conditions in the boundary integral formulation. Numerical experiments verify
the desired properties of the method and the expected accuracy for the
two-dimensional Poisson and Helmholtz equations with variable coefficients.

本文中，我们提出了一种名为 BIN-G 的新型边界积分网络，用于学习与域无关的 Green 函数。通过使用基于径向基函数的神经网络评估 BIN-G 中的 Green 函数，我们通过最小化 PDE 的残差以及预定测试函数的边界积分方程的均方误差来训练 BIN-G。通过利用 Green 函数的对称性并在 Green 函数奇点附近控制径向基函数核的细化，我们证明了我们的数值方案可以快速训练并准确评估具有变系数 PDE 的 Green 函数。所学习的 Green 函数与边界积分表述中的域几何、强制项和边界条件无关。数值实验证实了该方法的期望属性和对于具有变系数的二维 Poisson 方程和 Helmholtz 方程的预期精度。

学习椭圆型偏微分方程的独立领域格林函数

Learning Domain-Independent Green's Function For Elliptic Partial  Differential Equations

Any performance analysis based on stochastic simulation is subject to the
errors inherent in misspecifying the modeling assumptions, particularly the
input distributions. In situations with little support from data, we
investigate the use of worst-case analysis to analyze these errors, by
representing the partial, nonparametric knowledge of the input models via
optimization constraints. We study the performance and robustness guarantees of
this approach. We design and analyze a numerical scheme for solving a general
class of simulation objectives and uncertainty specifications. The key steps
involve a randomized discretization of the probability spaces, a simulable
unbiased gradient estimator using a nonparametric analog of the likelihood
ratio method, and a Frank-Wolfe (FW) variant of the stochastic approximation
(SA) method (which we call FWSA) run on the space of input probability
distributions. A convergence analysis for FWSA on non-convex problems is
provided. We test the performance of our approach via several numerical
examples.

利用最坏情况分析代替输入分布的部分、非参数化知识进行性能分析，为解决非随机问题提出数值方案，并进行收敛分析和数值案例测试。