Monotonic neural networks have recently been proposed as a way to define
invertible transformations. These transformations can be combined into powerful
autoregressive flows that have been shown to be universal approximators of
continuous probability distributions. Architectures that ensure monotonicity
typically enforce constraints on weights and activation functions, which
enables invertibility but leads to a cap on the expressiveness of the resulting
transformations. In this work, we propose the Unconstrained Monotonic Neural
Network (UMNN) architecture based on the insight that a function is monotonic
as long as its derivative is strictly positive. In particular, this latter
condition can be enforced with a free-form neural network whose only constraint
is the positiveness of its output. We evaluate our new invertible building
block within a new autoregressive flow (UMNN-MAF) and demonstrate its
effectiveness on density estimation experiments. We also illustrate the ability
of UMNNs to improve variational inference.

该研究提出了一种无限制单调神经网络（UMNN）架构，基于函数导数严格为正，进而在自回归流中实现密度估计实验，并展示了 UMNN 提高变分推断的能力。