We introduce SPLASH units, a class of learnable activation functions shown to simultaneously improve the accuracy of deep neural networks while also improving their robustness to adversarial attacks. SPLASH units have both a simple parameterization and maintain the ability to approximate a wide range of non-linear functions. SPLASH units are: 1) continuous; 2) grounded (f(0) = 0); 3) use symmetric hinges; and 4) the locations of the hinges are derived directly from the data (i.e. no learning required). Compared to nine other learned and fixed activation functions, including ReLU and its variants, SPLASH units show superior performance across three datasets (MNIST, CIFAR-10, and CIFAR-100) and four architectures (LeNet5, All-CNN, ResNet-20, and Network-in-Network). Furthermore, we show that SPLASH units significantly increase the robustness of deep neural networks to adversarial attacks. Our experiments on both black-box and open-box adversarial attacks show that commonly-used architectures, namely LeNet5, All-CNN, ResNet-20, and Network-in-Network, can be up to 31% more robust to adversarial attacks by simply using SPLASH units instead of ReLUs.

本文介绍了SPLASH unit这一种可学习的激活函数，该函数不仅能提高深度神经网络的准确性，还能提高其对抗攻击的鲁棒性。相较于其他九种常见的学习型和固定型激活函数，SPLASH units表现更优，能够适应各种非线性函数，可以使包括LeNet5、All-CNN、ResNet-20和Network-in-Network在内的常规神经网络对抗攻击的鲁棒性提高至多31％。

SPLASH: 可学习的激活函数用于提高准确性和对抗鲁棒性