BriefGPT.xyz
Feb, 2023
Moderate Adaptive Linear Units (MoLU)
Moderate Adaptive Linear Units (MoLU)
HTML
PDF
Hankyul Koh, Joon-hyuk Ko, Wonho Jhe
TL;DR
提出了一种新的高性能激活函数MoLU，该函数由基本函数构成，是光滑且在整个定义域上具有无限可微性，可以降低训练时间。
Abstract
We propose a new high-performance
activation function
, Moderate Adaptive Linear Units (
molu
), for the deep
neural network
. The
→