In this paper, we propose novel quaternion activation functions where we
modify either the quaternion magnitude or the phase, as an alternative to the
commonly used split activation functions. We define criteria that are relevant
for quaternion activation functions, and subsequently we propose our novel
activation functions based on this analysis. Instead of applying a known
activation function like the ReLU or Tanh on the quaternion elements
separately, these activation functions consider the quaternion properties and
respect the quaternion space $\mathbb{H}$. In particular, all quaternion
components are utilized to calculate all output components, carrying out the
benefit of the Hamilton product in e.g. the quaternion convolution to the
activation functions. The proposed activation functions can be incorporated in
arbitrary quaternion valued neural networks trained with gradient descent
techniques. We further discuss the derivatives of the proposed activation
functions where we observe beneficial properties for the activation functions
affecting the phase. Specifically, they prove to be sensitive on basically the
whole input range, thus improved gradient flow can be expected. We provide an
elaborate experimental evaluation of our proposed quaternion activation
functions including comparison with the split ReLU and split Tanh on two image
classification tasks using the CIFAR-10 and SVHN dataset. There, especially the
quaternion activation functions affecting the phase consistently prove to
provide better performance.

我们提出了新颖的四元数激活函数，其中修改了四元数的幅度或相位，作为常用分裂激活函数的替代。我们定义了四元数激活函数相关的准则，并基于该分析提出了我们的新颖激活函数。这些激活函数考虑了四元数的特性和四元数空间 H，通过利用所有四元数的分量计算所有输出分量，从而获得了四元数卷积中的 Hamilton 乘积的优势。我们进一步讨论了所提出的激活函数的导数，发现相位影响的激活函数具有有益的属性。我们对提出的四元数激活函数进行了详细的实验评估，并与基于分裂 ReLU 和分裂双曲正切的方法在 CIFAR-10 和 SVHN 数据集上进行了比较。在其中，尤其是影响相位的四元数激活函数始终提供更好的性能。