Structural modularity is a pervasive feature of biological neural networks,
which have been linked to several functional and computational advantages. Yet,
the use of modular architectures in artificial neural networks has been
relatively limited despite early successes. Here, we explore the performance
and functional dynamics of a modular network trained on a memory task via an
iterative growth curriculum. We find that for a given classical, non-modular
recurrent neural network (RNN), an equivalent modular network will perform
better across multiple metrics, including training time, generalizability, and
robustness to some perturbations. We further examine how different aspects of a
modular network's connectivity contribute to its computational capability. We
then demonstrate that the inductive bias introduced by the modular topology is
strong enough for the network to perform well even when the connectivity within
modules is fixed and only the connections between modules are trained. Our
findings suggest that gradual modular growth of RNNs could provide advantages
for learning increasingly complex tasks on evolutionary timescales, and help
build more scalable and compressible artificial networks.

通过渐进式的模块化增长，研究表明，模块化网络在记忆任务上具有更好的性能、更强的泛化能力和对某些扰动的鲁棒性，即使只有模块之间的连接被训练，结构拓扑引入的归纳偏置足够强大，使得网络在固定了模块内连接的情况下仍能表现良好，并且逐渐模块化增长的循环神经网络能够在进化时间尺度上学习越来越复杂的任务，并且有助于构建更可扩展且可压缩的人工网络。

层次网络的模块化增长：高效、通用、稳健的课程学习

Modular Growth of Hierarchical Networks: Efficient, General, and Robust  Curriculum Learning

Appearance-based detectors achieve remarkable performance on common scenes,
but tend to fail for scenarios lack of training data. Geometric motion
segmentation algorithms, however, generalize to novel scenes, but have yet to
achieve comparable performance to appearance-based ones, due to noisy motion
estimations and degenerate motion configurations. To combine the best of both
worlds, we propose a modular network, whose architecture is motivated by a
geometric analysis of what independent object motions can be recovered from an
egomotion field. It takes two consecutive frames as input and predicts
segmentation masks for the background and multiple rigidly moving objects,
which are then parameterized by 3D rigid transformations. Our method achieves
state-of-the-art performance for rigid motion segmentation on KITTI and Sintel.
The inferred rigid motions lead to a significant improvement for depth and
scene flow estimation. At the time of submission, our method ranked 1st on
KITTI scene flow leaderboard, out-performing the best published method (scene
flow error: 4.89% vs 6.31%).

提出了一种模块化网络，利用几何分析独立对象运动能从码流场中恢复，利用 3D 刚体转换对背景和多个刚体运动物体进行分割掩模并获得刚性运动分割的最佳性能，从而显着改善深度和场景流估计。