In this paper, we derive a PAC-Bayes bound on the generalisation gap, in a
supervised time-series setting for a special class of discrete-time non-linear
dynamical systems. This class includes stable recurrent neural networks (RNN),
and the motivation for this work was its application to RNNs. In order to
achieve the results, we impose some stability constraints, on the allowed
models. Here, stability is understood in the sense of dynamical systems. For
RNNs, these stability conditions can be expressed in terms of conditions on the
weights. We assume the processes involved are essentially bounded and the loss
functions are Lipschitz. The proposed bound on the generalisation gap depends
on the mixing coefficient of the data distribution, and the essential supremum
of the data. Furthermore, the bound converges to zero as the dataset size
increases. In this paper, we 1) formalize the learning problem, 2) derive a
PAC-Bayesian error bound for such systems, 3) discuss various consequences of
this error bound, and 4) show an illustrative example, with discussions on
computing the proposed bound. Unlike other available bounds the derived bound
holds for non i.i.d. data (time-series) and it does not grow with the number of
steps of the RNN.

在这篇论文中，我们推导了一个 PAC-Bayes 界限，用于一类特殊的离散时间非线性动力系统的监督时间序列设置。这个类别包括稳定的递归神经网络（RNN），而这项工作的动机就是应用于 RNN。我们在允许的模型上施加一些稳定性约束，这里的稳定性是以动力系统的概念来理解的。对于 RNN，这些稳定性条件可以表示为关于权重的条件。我们假设所涉及的过程在本质上是有界的，并且损失函数是利普希茨的。所提出的对于泛化差距的界限依赖于数据分布的混合系数和数据的本质上最大值。此外，随着数据集大小的增加，这个界限收敛于零。在这篇论文中，我们 1）正式化了学习问题，2）为这类系统推导了一个 PAC-Bayesian 误差界限，3）讨论了这个误差界限的各种结果，以及 4）展示了一个说明性例子，并讨论了计算所提出的界限的方法。与其他可用的界限不同，这个推导的界限适用于非独立同分布的数据（时序数据），并且它不随 RNN 的步骤数增长。