We present experimental results highlighting two key differences resulting
from the choice of training algorithm for two-layer neural networks. The
spectral bias of neural networks is well known, while the spectral bias
dependence on the choice of training algorithm is less studied. Our experiments
demonstrate that an adaptive random Fourier features algorithm (ARFF) can yield
a spectral bias closer to zero compared to the stochastic gradient descent
optimizer (SGD). Additionally, we train two identically structured classifiers,
employing SGD and ARFF, to the same accuracy levels and empirically assess
their robustness against adversarial noise attacks.

我们通过实验结果发现，训练算法选择引起了两层神经网络的两个关键差异。神经网络的谱偏倚是众所周知的，而谱偏倚对于训练算法的选择依赖则研究较少。我们的实验表明，自适应随机傅里叶特征算法（ARFF）相比于随机梯度下降优化器（SGD）可以产生更接近于零的谱偏倚。此外，我们用 SGD 和 ARFF 训练两个结构相同的分类器，将它们的准确度提升到相同水平，并通过实验证明它们对抗敌对噪声攻击的鲁棒性。