Deep neural networks, despite their success in numerous applications, often function without established theoretical foundations. In this paper, we bridge this gap by drawing parallels between deep learning and classical numerical analysis. By framing neural networks as operators with fixed points representing desired solutions, we develop a theoretical framework grounded in iterative methods for operator equations. Under defined conditions, we present convergence proofs based on fixed point theory. We demonstrate that popular architectures, such as diffusion models and AlphaFold, inherently employ iterative operator learning. Empirical assessments highlight that performing iterations through network operators improves performance. We also introduce an iterative graph neural network, PIGN, that further demonstrates benefits of iterations. Our work aims to enhance the understanding of deep learning by merging insights from numerical analysis, potentially guiding the design of future networks with clearer theoretical underpinnings and improved performance.

通过将神经网络定位为具有固定点表示所需解的算子，我们以数值分析为基础，建立了一个以迭代方法为基础的理论框架，以理论证明为基础，我们演示了流行的架构，如扩散模型和AlphaFold，本质上使用了迭代算子学习，经验评估表明通过网络算子进行迭代可以提高性能。我们还介绍了一个迭代图神经网络PIGN，进一步展示了迭代的好处。我们的工作旨在通过融合数值分析的洞察力，从而提升深度学习的理解，潜在地指导设计具有更清晰理论基础和改进性能的未来网络。

运算学习遇见数值分析：通过迭代方法改进神经网络