Directly parameterizing and learning gradients of functions has widespread
significance, with specific applications in optimization, generative modeling,
and optimal transport. This paper introduces gradient networks (GradNets):
novel neural network architectures that parameterize gradients of various
function classes. GradNets exhibit specialized architectural constraints that
ensure correspondence to gradient functions. We provide a comprehensive GradNet
design framework that includes methods for transforming GradNets into monotone
gradient networks (mGradNets), which are guaranteed to represent gradients of
convex functions. We establish the approximation capabilities of the proposed
GradNet and mGradNet. Our results demonstrate that these networks universally
approximate the gradients of (convex) functions. Furthermore, these networks
can be customized to correspond to specific spaces of (monotone) gradient
functions, including gradients of transformed sums of (convex) ridge functions.
Our analysis leads to two distinct GradNet architectures, GradNet-C and
GradNet-M, and we describe the corresponding monotone versions, mGradNet-C and
mGradNet-M. Our empirical results show that these architectures offer efficient
parameterizations and outperform popular methods in gradient field learning
tasks.

本文介绍了梯度网络（GradNets）：一种参数化各类函数梯度的新型神经网络架构，通过增加特定约束确保与梯度函数的对应性。我们提供了一个全面的 GradNet 设计框架，包括将 GradNets 转化为保证表示凸函数梯度的单调梯度网络（mGradNets）的方法。我们证明了所提出的 GradNet 和 mGradNet 的近似能力，并且实验结果表明这些架构在梯度场学习任务中提供了高效的参数化，并且优于流行的方法。