This paper introduces Non-Autonomous Input-Output Stable Network(NAIS-Net), a
very deep architecture where each stacked processing block is derived from a
time-invariant non-autonomous dynamical system. Non-autonomy is implemented by
skip connections from the block input to each of the unrolled processing stages
and allows stability to be enforced so that blocks can be unrolled adaptively
to a pattern-dependent processing depth. NAIS-Net induces non-trivial,
Lipschitz input-output maps, even for an infinite unroll length. We prove that
the network is globally asymptotically stable so that for every initial
condition there is exactly one input-dependent equilibrium assuming $tanh$
units, and incrementally stable for ReL units. An efficient implementation that
enforces the stability under derived conditions for both fully-connected and
convolutional layers is also presented. Experimental results show how NAIS-Net
exhibits stability in practice, yielding a significant reduction in
generalization gap compared to ResNets.

本文介绍了非自治输入输出稳定网络（NAIS-Net），这是一种非常深的架构，每个堆叠处理块都来自于时间不变的非自治动力系统，通过跳跃连接实现非自治，从而允许强制实现稳定性，使得块能够自适应地展开到具体的模式处理深度，该网络具有非平凡的利普希茨输入输出映射，全局渐进稳定，实验结果表明，与 ResNets 相比，NAIS-Net 在实践中展现出了稳定性，使得泛化差距显著降低。