Sign-based stochastic methods have gained attention due to their ability to achieve robust performance despite using only the sign information for parameter updates. However, the current convergence analysis of sign-based methods relies on the strong assumptions of first-order gradient Lipschitz and second-order gradient Lipschitz, which may not hold in practical tasks like deep neural network training that involve high non-smoothness. In this paper, we revisit sign-based methods and analyze their convergence under more realistic assumptions of first- and second-order smoothness. We first establish the convergence of the sign-based method under weak first-order Lipschitz. Motivated by the weak first-order Lipschitz, we propose a relaxed second-order condition that still allows for nonconvex acceleration in sign-based methods. Based on our theoretical results, we gain insights into the computational advantages of the recently developed LION algorithm. In distributed settings, we prove that this nonconvex acceleration persists with linear speedup in the number of nodes, when utilizing fast communication compression gossip protocols. The novelty of our theoretical results lies in that they are derived under much weaker assumptions, thereby expanding the provable applicability of sign-based algorithms to a wider range of problems.

基于信号的方法是一种能够在参数更新时只使用符号信息而实现稳健性能的方法。本文重新审视了基于信号的方法，并在更现实的一阶和二阶平滑性的假设下分析了其收敛性。基于我们的理论结果，我们对最近开发的LION算法的计算优势有了深入的了解。在分布式环境中，我们证明了当利用快速通信压缩传播协议时，这种非凸加速度随着节点数量的线性加速性仍然存在。我们的理论结果的新颖之处在于它们是在更弱的假设下得出的，从而扩展了基于信号算法可适用于更广泛问题范围的可证明应用性。

重新思考SIGN训练: 可证明的非凸加速，无需一阶和二阶梯度Lipschitz