In i-theory a typical layer of a hierarchical architecture consists of HW
modules pooling the dot products of the inputs to the layer with the
transformations of a few templates under a group. Such layers include as
special cases the convolutional layers of Deep Convolutional Networks (DCNs) as
well as the non-convolutional layers (when the group contains only the
identity). Rectifying nonlinearities -- which are used by present-day DCNs --
are one of the several nonlinearities admitted by i-theory for the HW module.
We discuss here the equivalence between group averages of linear combinations
of rectifying nonlinearities and an associated kernel. This property implies
that present-day DCNs can be exactly equivalent to a hierarchy of kernel
machines with pooling and non-pooling layers. Finally, we describe a conjecture
for theoretically understanding hierarchies of such modules. A main consequence
of the conjecture is that hierarchies of trained HW modules minimize memory
requirements while computing a selective and invariant representation.

该论文讨论了 i-theory 中典型层次结构的构成，提出现代 DCNs 可以与具有池化和非池化层次的核机器的层级架构完全等效，最终提出高效的结构猜想。