Activation functions are crucial in deep learning models since they introduce
non-linearity into the networks, allowing them to learn from errors and make
adjustments, which is essential for learning complex patterns. The essential
purpose of activation functions is to transform unprocessed input signals into
significant output activations, promoting information transmission throughout
the neural network. In this study, we propose a new activation function called
Sqish, which is a non-monotonic and smooth function and an alternative to
existing ones. We showed its superiority in classification, object detection,
segmentation tasks, and adversarial robustness experiments. We got an 8.21%
improvement over ReLU on the CIFAR100 dataset with the ShuffleNet V2 model in
the FGSM adversarial attack. We also got a 5.87% improvement over ReLU on image
classification on the CIFAR100 dataset with the ShuffleNet V2 model.

本研究提出了一种名为 Sqish 的新型激活函数，作为现有激活函数的替代品，我们展示了它在分类、目标检测、分割任务和对抗性鲁棒性实验中的优越性，在 CIFAR100 数据集上，使用 ShuffleNet V2 模型在 FGSM 对抗攻击中，相较于 ReLU 取得了 8.21% 的改进，并且在 CIFAR100 数据集上，使用 ShuffleNet V2 模型进行图像分类，相较于 ReLU 取得了 5.87% 的改进。