We present a novel algorithm for training deep neural networks in supervised
(classification and regression) and unsupervised (reinforcement learning)
scenarios. This algorithm combines the standard stochastic gradient descent and
the gradient clipping method. The output layer is updated using clipped
gradients, the rest of the neural network is updated using standard gradients.
Updating the output layer using clipped gradient stabilizes it. We show that
the remaining layers are automatically stabilized provided the neural network
is only composed of squashing (compact range) activations. We also present a
novel squashing activation function - it is obtained by modifying a Gaussian
Error Linear Unit (GELU) to have compact range - we call it Truncated GELU
(tGELU). Unlike other squashing activations, such as sigmoid, the range of
tGELU can be explicitly specified. As a consequence, the problem of vanishing
gradients that arise due to a small range, e.g., in the case of a sigmoid
activation, is eliminated. We prove that a NN composed of squashing activations
(tGELU, sigmoid, etc.), when updated using the algorithm presented herein, is
numerically stable and has consistent performance (low variance). The theory is
supported by extensive experiments. Within reinforcement learning, as a
consequence of our study, we show that target networks in Deep Q-Learning can
be omitted, greatly speeding up learning and alleviating memory requirements.
Cross-entropy based classification algorithms that suffer from high variance
issues are more consistent when trained using our framework. One symptom of
numerical instability in training is the high variance of the neural network
update values. We show, in theory and through experiments, that our algorithm
updates have low variance, and the training loss reduces in a smooth manner.

介绍了一种针对深度神经网络进行有监督训练、无监督训练以及强化学习的新算法，运用标准随机梯度下降法和梯度剪裁法对神经网络进行更新，提出了一种稳定输出层且范围明确的新型激活函数，得出结论：神经网络的更新值稳定，训练更加精确与流畅。