TL;DR本文提出了一种名为 MBA 的多偏置非线性激活层,它采用多重阈值对卷积核的响应进行解耦,从而在特征空间中生成更多的模式,并在低计算成本下选择不同幅度范围内的响应来形成更丰富的表示。此简单而有效的方案在多个基准测试中实现了最先进的性能。
Abstract
As a widely used non-linear activation, rectified linear unit (ReLU)
separates noise and signal in a feature map by learning a threshold or bias.
However, we argue that the classification of noise and signal not