Natural target functions and tasks typically exhibit hierarchical modularity
- they can be broken down into simpler sub-functions that are organized in a
hierarchy. Such sub-functions have two important features: they have a distinct
set of inputs (input-separability) and they are reused as inputs higher in the
hierarchy (reusability). Previous studies have established that hierarchically
modular neural networks, which are inherently sparse, offer benefits such as
learning efficiency, generalization, multi-task learning, and transferability.
However, identifying the underlying sub-functions and their hierarchical
structure for a given task can be challenging. The high-level question in this
work is: if we learn a task using a sufficiently deep neural network, how can
we uncover the underlying hierarchy of sub-functions in that task? As a
starting point, we examine the domain of Boolean functions, where it is easier
to determine whether a task is hierarchically modular. We propose an approach
based on iterative unit and edge pruning (during training), combined with
network analysis for module detection and hierarchy inference. Finally, we
demonstrate that this method can uncover the hierarchical modularity of a wide
range of Boolean functions and two vision tasks based on the MNIST digits
dataset.

本研究提出了一种基于逐层单元剪枝和网络分析的方法，用于揭示任务中底层次模块的分层结构，结果表明该方法可以适用于布尔函数和视觉任务。