Modeling long-range dependencies in sequential data is a crucial step in
sequence learning. A recently developed model, the Structured State Space (S4),
demonstrated significant effectiveness in modeling long-range sequences.
However, It is unclear whether the success of S4 can be attributed to its
intricate parameterization and HiPPO initialization or simply due to State
Space Models (SSMs). To further investigate the potential of the deep SSMs, we
start with exponential smoothing (ETS), a simple SSM, and propose a stacked
architecture by directly incorporating it into an element-wise MLP. We augment
simple ETS with additional parameters and complex field to reduce the inductive
bias. Despite increasing less than 1\% of parameters of element-wise MLP, our
models achieve comparable results to S4 on the LRA benchmark.

建模序列数据中的长程依赖是序列学习中关键的一步，本研究提出了一种新模型 Structured State Space（S4），通过结构化状态空间模型在建模长程序列方面表现出了显著的有效性。与 S4 相比，本研究采用指数平滑（ETS）作为简单的状态空间模型，并将其直接融入逐元素 MLP 中，通过添加额外参数和复杂域来减少归纳偏差，虽然我们的模型仅比逐元素 MLP 增加了不到 1% 的参数，但在 LRA 基准测试上取得了可比较的结果。

将指数平滑法纳入 MLP：一种简单而有效的序列模型

Incorporating Exponential Smoothing into MLP: A Simple but Effective  Sequence Model

The FlowNet demonstrated that optical flow estimation can be cast as a
learning problem. However, the state of the art with regard to the quality of
the flow has still been defined by traditional methods. Particularly on small
displacements and real-world data, FlowNet cannot compete with variational
methods. In this paper, we advance the concept of end-to-end learning of
optical flow and make it work really well. The large improvements in quality
and speed are caused by three major contributions: first, we focus on the
training data and show that the schedule of presenting data during training is
very important. Second, we develop a stacked architecture that includes warping
of the second image with intermediate optical flow. Third, we elaborate on
small displacements by introducing a sub-network specializing on small motions.
FlowNet 2.0 is only marginally slower than the original FlowNet but decreases
the estimation error by more than 50%. It performs on par with state-of-the-art
methods, while running at interactive frame rates. Moreover, we present faster
variants that allow optical flow computation at up to 140fps with accuracy
matching the original FlowNet.

本文提出了一种基于端到端学习的光流估计方法，该方法通过数据给特定学习任务提供重要信息，并且采用了堆叠式架构及对小运动专门优化的子网络，使得计算速度得到了提升，同时精度与业界领先的方法相当；相比于原始的 FlowNet，FlowNet 2.0 计算速度基本不降反升，估计误差降低了超过 50%。