Diagrams matter. Unfortunately, the deep learning community has no standard
method for diagramming architectures. The current combination of linear algebra
notation and ad-hoc diagrams fails to offer the necessary precision to
understand architectures in all their detail. However, this detail is critical
for faithful implementation, mathematical analysis, further innovation, and
ethical assurances. I present neural circuit diagrams, a graphical language
tailored to the needs of communicating deep learning architectures. Neural
circuit diagrams naturally keep track of the changing arrangement of data,
precisely show how operations are broadcast over axes, and display the critical
parallel behavior of linear operations. A lingering issue with existing
diagramming methods is the inability to simultaneously express the detail of
axes and the free arrangement of data, which neural circuit diagrams solve.
Their compositional structure is analogous to code, creating a close
correspondence between diagrams and implementation.
In this work, I introduce neural circuit diagrams for an audience of machine
learning researchers. After introducing neural circuit diagrams, I cover a host
of architectures to show their utility and breed familiarity. This includes the
transformer architecture, convolution (and its difficult-to-explain
extensions), residual networks, the U-Net, and the vision transformer. I
include a Jupyter notebook that provides evidence for the close correspondence
between diagrams and code. Finally, I examine backpropagation using neural
circuit diagrams. I show their utility in providing mathematical insight and
analyzing algorithms' time and space complexities.

使用神经电路图图示化深度学习架构，展示架构的细节、数据的排列方式以及线性操作的并行行为，并且能够提供数学洞察力和算法的时间和空间复杂度分析。