To push forward the important emerging research field surrounding multi-area
recurrent neural networks (RNNs), we expand theoretically and empirically on
the provably stable RNNs of RNNs introduced by Kozachkov et al. in "RNNs of
RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural
Networks". We prove relaxed stability conditions for salient special cases of
this architecture, most notably for a global workspace modular structure. We
then demonstrate empirical success for Global Workspace Sparse Combo Nets with
a small number of trainable parameters, not only through strong overall test
performance but also greater resilience to removal of individual subnetworks.
These empirical results for the global workspace inter-area topology are
contingent on stability preservation, highlighting the relevance of our
theoretical work for enabling modular RNN success. Further, by exploring
sparsity in the connectivity structure between different subnetwork modules
more broadly, we improve the state of the art performance for stable RNNs on
benchmark sequence processing tasks, thus underscoring the general utility of
specialized graph structures for multi-area RNNs.

我们通过扩展 Kozachkov 等人在《RNNs of RNNs: Recursive Construction of Stable Assemblies of Recurrent Neural Networks》中引入的 RNNs of RNNs 的稳定性证明，为该体系结构的显著特殊情况，尤其是全局工作空间模块化结构，提供了理论上和实证上的支持；通过探索不同子网络模块之间的连接稀疏性结构，我们改进了基准序列处理任务上稳定 RNNs 的性能，强调了专门的图结构对于多区域 RNNs 的一般实用性。