In the rapidly advancing domain of deep learning optimization, this paper
unveils the StochGradAdam optimizer, a novel adaptation of the well-regarded
Adam algorithm. Central to StochGradAdam is its gradient sampling technique.
This method not only ensures stable convergence but also leverages the
advantages of selective gradient consideration, fostering robust training by
potentially mitigating the effects of noisy or outlier data and enhancing the
exploration of the loss landscape for more dependable convergence. In both
image classification and segmentation tasks, StochGradAdam has demonstrated
superior performance compared to the traditional Adam optimizer. By judiciously
sampling a subset of gradients at each iteration, the optimizer is optimized
for managing intricate models. The paper provides a comprehensive exploration
of StochGradAdam's methodology, from its mathematical foundations to bias
correction strategies, heralding a promising advancement in deep learning
training techniques.

该研究论文介绍了深度学习优化领域中的 StochGradAdam 优化器，它是 Adam 算法的一种新变体，核心是梯度采样技术。该方法不仅确保了稳定的收敛性，还充分利用了选择性梯度考虑的优势，通过减轻嘈杂或异常数据的影响以及增强损失函数空间的探索，提高了可靠性收敛。在图像分类和分割任务中，StochGradAdam 优化器表现出优越的性能，相较于传统的 Adam 优化器。通过在每次迭代中精确采样一部分梯度，该优化器被优化用于管理复杂模型。该论文全面探讨了 StochGradAdam 的方法论，从数学基础到偏差校正策略，为深度学习训练技术的有望进展铺平了道路。