We show the sup-norm convergence of deep neural network estimators with a novel adversarial training scheme. For the nonparametric regression problem, it has been shown that an estimator using deep neural networks can achieve better performances in the sense of the $L2$-norm. In contrast, it is difficult for the neural estimator with least-squares to achieve the sup-norm convergence, due to the deep structure of neural network models. In this study, we develop an adversarial training scheme and investigate the sup-norm convergence of deep neural network estimators. First, we find that ordinary adversarial training makes neural estimators inconsistent. Second, we show that a deep neural network estimator achieves the optimal rate in the sup-norm sense by the proposed adversarial training with correction. We extend our adversarial training to general setups of a loss function and a data-generating function. Our experiments support the theoretical findings.

我们展示了具有新颖对抗训练方案的深度神经网络估计器的上限范数收敛。对于非参数回归问题，已经证明使用深度神经网络的估计器可以在$L2$-范数意义上取得更好的性能。相比之下，由于神经网络模型的深度结构，最小二乘神经估计器很难实现上限范数的收敛。在本研究中，我们开发了一种对抗性训练方案，并研究了深度神经网络估计器的上限范数收敛。首先，我们发现普通的对抗性训练使得神经估计器不一致。其次，我们证明了通过所提出的带校正的对抗性训练，深度神经网络估计器在上限范数意义上实现了最优速率。我们将对抗性训练扩展到损失函数和数据生成函数的一般设置。我们的实验证实了理论发现。

非参数回归问题中深度神经网络估计器的超范数收敛与对抗训练