Recently, deep learning-based compressed sensing (CS) has achieved great
success in reducing the sampling and computational cost of sensing systems and
improving the reconstruction quality. These approaches, however, largely
overlook the issue of the computational cost; they rely on complex structures
and task-specific operator designs, resulting in extensive storage and high
energy consumption in CS imaging systems. In this paper, we propose a
lightweight but effective deep neural network based on recurrent learning to
achieve a sustainable CS system; it requires a smaller number of parameters but
obtains high-quality reconstructions. Specifically, our proposed network
consists of an initial reconstruction sub-network and a residual reconstruction
sub-network. While the initial reconstruction sub-network has a hierarchical
structure to progressively recover the image, reducing the number of
parameters, the residual reconstruction sub-network facilitates recurrent
residual feature extraction via recurrent learning to perform both feature
fusion and deep reconstructions across different scales. In addition, we also
demonstrate that, after the initial reconstruction, feature maps with reduced
sizes are sufficient to recover the residual information, and thus we achieved
a significant reduction in the amount of memory required. Extensive experiments
illustrate that our proposed model can achieve a better reconstruction quality
than existing state-of-the-art CS algorithms, and it also has a smaller number
of network parameters than these algorithms. Our source codes are available at:
this https URL

该论文提出一种轻量级深度神经网络，基于逐渐回复图像的分层结构和使用循环学习进行深度重构，以达到高质量重构图像的同时降低计算、能源和存储成本，模型的性能优于现有压缩感知算法。

一种轻量级的递归学习网络，用于可持续的压缩感知

A Lightweight Recurrent Learning Network for Sustainable Compressed  Sensing

We cast Amari's natural gradient in statistical learning as a specific case
of Kalman filtering. Namely, applying an extended Kalman filter to estimate a
fixed unknown parameter of a probabilistic model from a series of observations,
is rigorously equivalent to estimating this parameter via an online stochastic
natural gradient descent on the log-likelihood of the observations.
In the i.i.d. case, this relation is a consequence of the "information
filter" phrasing of the extended Kalman filter. In the recurrent (state space,
non-i.i.d.) case, we prove that the joint Kalman filter over states and
parameters is a natural gradient on top of real-time recurrent learning (RTRL),
a classical algorithm to train recurrent models.
This exact algebraic correspondence provides relevant interpretations for
natural gradient hyperparameters such as learning rates or initialization and
regularization of the Fisher information matrix.

本文证明了卡尔曼滤波算法可以作为自然梯度下降在参数评估上的应用，尤其是在递归学习等应用中。

在线自然梯度作为卡尔曼滤波器

Online Natural Gradient as a Kalman Filter

We systematically explored a spectrum of normalization algorithms related to
Batch Normalization (BN) and propose a generalized formulation that
simultaneously solves two major limitations of BN: (1) online learning and (2)
recurrent learning. Our proposal is simpler and more biologically-plausible.
Unlike previous approaches, our technique can be applied out of the box to all
learning scenarios (e.g., online learning, batch learning, fully-connected,
convolutional, feedforward, recurrent and mixed --- recurrent and
convolutional) and compare favorably with existing approaches. We also propose
Lp Normalization for normalizing by different orders of statistical moments. In
particular, L1 normalization is well-performing, simple to implement, fast to
compute, more biologically-plausible and thus ideal for GPU or hardware
implementations.

提出了一个更简单、更符合生物学规律的通用归一化算法，可同时解决批归一化的两个主要限制：在线学习和循环学习，并提出了用不同统计矩阶数来进行归一化的 Lp 归一化方法，特别是 L1 归一化，其性能表现良好且计算速度快，更加符合生物学规律，因此非常适合 GPU 或硬件实现。