Despite the remarkable capabilities of deep neural networks in image
recognition, the dependence on activation functions remains a largely
unexplored area and has yet to be eliminated. On the other hand, Polynomial
Networks is a class of models that does not require activation functions, but
have yet to perform on par with modern architectures. In this work, we aim
close this gap and propose MONet, which relies solely on multilinear operators.
The core layer of MONet, called Mu-Layer, captures multiplicative interactions
of the elements of the input token. MONet captures high-degree interactions of
the input elements and we demonstrate the efficacy of our approach on a series
of image recognition and scientific computing benchmarks. The proposed model
outperforms prior polynomial networks and performs on par with modern
architectures. We believe that MONet can inspire further research on models
that use entirely multilinear operations.

提出了一种仅依赖于多线性操作、名为 Mu-Layer 的核心层的模型 MONet，在图像识别和科学计算基准测试中表现优异，超越了之前的多项式网络，并与现代架构相当，有望激发进一步研究纯粹使用多线性操作的模型。