The application of the deep learning model in classification plays an
important role in the accurate detection of the target objects. However, the
accuracy is affected by the activation function in the hidden and output layer.
In this paper, an activation function called TaLU, which is a combination of
Tanh and Rectified Linear Units (ReLU), is used to improve the prediction. ReLU
activation function is used by many deep learning researchers for its
computational efficiency, ease of implementation, intuitive nature, etc.
However, it suffers from a dying gradient problem. For instance, when the input
is negative, its output is always zero because its gradient is zero. A number
of researchers used different approaches to solve this issue. Some of the most
notable are LeakyReLU, Softplus, Softsign, Elu, ThresholdedReLU, etc. This
research developed TaLU, a modified activation function combining Tanh and
ReLU, which mitigates the dying gradient problem of ReLU. The deep learning
model with the proposed activation function was tested on MNIST and CIFAR-10,
and it outperforms ReLU and some other studied activation functions in terms of
accuracy(from 0\% upto 6\% in most cases, when used with Batch Normalization
and a reasonable learning rate).

本文研究使用一种名为 TaLU 的激活函数来提高深度学习模型的分类准确度，该激活函数是 Tanh 和 ReLU 的组合，能够缓解 ReLU 的梯度消失问题，并在 MNIST 和 CIFAR-10 数据集上实现了 0％至 6％不等的准确度提高。