Recent efforts in applying implicit networks to solve inverse problems in
imaging have achieved competitive or even superior results when compared to
feedforward networks. These implicit networks only require constant memory
during backpropagation, regardless of the number of layers. However, they are
not necessarily easy to train. Gradient calculations are computationally
expensive because they require backpropagating through a fixed point. In
particular, this process requires solving a large linear system whose size is
determined by the number of features in the fixed point iteration. This paper
explores a recently proposed method, Jacobian-free Backpropagation (JFB), a
backpropagation scheme that circumvents such calculation, in the context of
image deblurring problems. Our results show that JFB is comparable against
fine-tuned optimization schemes, state-of-the-art (SOTA) feedforward networks,
and existing implicit networks at a reduced computational cost.

通过在图像去模糊问题的背景下探索最近提出的无雅可比反向传播（Jacobian-free Backpropagation，JFB）方法，本文展示了 JFB 在减少计算成本方面与优化方案、最先进的前馈网络以及现有的隐式网络具有可比性。

使用无雅各比传递正向传播训练隐式网络进行图像去模糊

Training Implicit Networks for Image Deblurring using Jacobian-Free  Backpropagation

A promising trend in deep learning replaces traditional feedforward networks
with implicit networks. Unlike traditional networks, implicit networks solve a
fixed point equation to compute inferences. Solving for the fixed point varies
in complexity, depending on provided data and an error tolerance. Importantly,
implicit networks may be trained with fixed memory costs in stark contrast to
feedforward networks, whose memory requirements scale linearly with depth.
However, there is no free lunch -- backpropagation through implicit networks
often requires solving a costly Jacobian-based equation arising from the
implicit function theorem. We propose Jacobian-Free Backpropagation (JFB), a
fixed-memory approach that circumvents the need to solve Jacobian-based
equations. JFB makes implicit networks faster to train and significantly easier
to implement, without sacrificing test accuracy. Our experiments show implicit
networks trained with JFB are competitive with feedforward networks and prior
implicit networks given the same number of parameters.

本研究提出一种名为 Jacobian-Free Backpropagation 的算法，用于优化隐式网络的反向传播过程，以节省训练时间和实现难度，并在测试精度方面保持与前沿水平相当，同时仍能满足固定内存成本的需求。