Automatically extracting the geometric content from the hundreds of thousands
of diagrams drawn in historical manuscripts would enable historians to study
the diffusion of astronomical knowledge on a global scale. However,
state-of-the-art vectorization methods, often designed to tackle modern data,
are not adapted to the complexity and diversity of historical astronomical
diagrams. Our contribution is thus twofold. First, we introduce a unique
dataset of 303 astronomical diagrams from diverse traditions, ranging from the
XIIth to the XVIIIth century, annotated with more than 3000 line segments,
circles and arcs. Second, we develop a model that builds on DINO-DETR to enable
the prediction of multiple geometric primitives. We show that it can be trained
solely on synthetic data and accurately predict primitives on our challenging
dataset. Our approach widely improves over the LETR baseline, which is
restricted to lines, by introducing a meaningful parametrization for multiple
primitives, jointly training for detection and parameter refinement, using
deformable attention and training on rich synthetic data. Our dataset and code
are available on our webpage.

从历史手稿中自动提取几十万个图表中的几何内容将使历史学家能够在全球范围内研究天文知识的传播。我们提供了一个独特的数据集，包含 303 个来自不同传统的天文图表，从 12 世纪到 18 世纪，注释了 3000 多个线段、圆和弧。我们还开发了一个模型，该模型基于 DINO-DETR，可以预测多个几何基元。我们展示了该模型可以仅通过合成数据进行训练，并准确地预测我们具有挑战性的数据集上的基元。我们的方法在多个基元的意义化参数化、检测和参数细化的联合训练以及使用丰富的合成数据进行训练方面明显改进了 LETR 基准。我们的数据集和代码可在我们的网页上获取。

历史天文图解的几何基元分解

Historical Astronomical Diagrams Decomposition in Geometric Primitives

Diagrams matter. Unfortunately, the deep learning community has no standard
method for diagramming architectures. The current combination of linear algebra
notation and ad-hoc diagrams fails to offer the necessary precision to
understand architectures in all their detail. However, this detail is critical
for faithful implementation, mathematical analysis, further innovation, and
ethical assurances. I present neural circuit diagrams, a graphical language
tailored to the needs of communicating deep learning architectures. Neural
circuit diagrams naturally keep track of the changing arrangement of data,
precisely show how operations are broadcast over axes, and display the critical
parallel behavior of linear operations. A lingering issue with existing
diagramming methods is the inability to simultaneously express the detail of
axes and the free arrangement of data, which neural circuit diagrams solve.
Their compositional structure is analogous to code, creating a close
correspondence between diagrams and implementation.
In this work, I introduce neural circuit diagrams for an audience of machine
learning researchers. After introducing neural circuit diagrams, I cover a host
of architectures to show their utility and breed familiarity. This includes the
transformer architecture, convolution (and its difficult-to-explain
extensions), residual networks, the U-Net, and the vision transformer. I
include a Jupyter notebook that provides evidence for the close correspondence
between diagrams and code. Finally, I examine backpropagation using neural
circuit diagrams. I show their utility in providing mathematical insight and
analyzing algorithms' time and space complexities.

使用神经电路图图示化深度学习架构，展示架构的细节、数据的排列方式以及线性操作的并行行为，并且能够提供数学洞察力和算法的时间和空间复杂度分析。

神经电路图：深度学习架构的通讯、实现和分析的稳健图示

Neural Circuit Diagrams: Robust Diagrams for the Communication,  Implementation, and Analysis of Deep Learning Architectures

Can machines think? Since Alan Turing asked this question in 1950, nobody is
able to give a direct answer, due to the lack of solid mathematical foundations
for general intelligence. In this paper, we introduce a categorical framework
towards this goal, consisting of four components: the sensor, world category,
planner with objectives, and actor. By leveraging category theory, many
important notions in general intelligence can be rigorously defined and
analyzed. For instance, we introduce the concept of self-state awareness as a
categorical analogy for self-consciousness and provide algorithms for learning
and evaluating it. For communication with other agents, we propose to use
diagrams that capture the exact representation of the context, instead of using
natural languages. Additionally, we demonstrate that by designing the
objectives as the output of function over self-state, the model's
human-friendliness is guaranteed. Most importantly, our framework naturally
introduces various constraints based on categorical invariance that can serve
as the alignment signals for training a model that fits into the framework.

本文提出了一个基于范畴论的分类框架，其中介绍了自我状态意识的概念作为自我意识的类比，并使用图表进行代理间通信，并通过将目标设计为自我状态函数的输出来保证模型的友好性。

一种一般智能的分类框架

A Categorical Framework of General Intelligence

Diagrams often depict complex phenomena and serve as a good test bed for
visual and textual reasoning. However, understanding diagrams using natural
image understanding approaches requires large training datasets of diagrams,
which are very hard to obtain. Instead, this can be addressed as a matching
problem either between labeled diagrams, images or both. This problem is very
challenging since the absence of significant color and texture renders local
cues ambiguous and requires global reasoning. We consider the problem of
one-shot part labeling: labeling multiple parts of an object in a target image
given only a single source image of that category. For this set-to-set matching
problem, we introduce the Structured Set Matching Network (SSMN), a structured
prediction model that incorporates convolutional neural networks. The SSMN is
trained using global normalization to maximize local match scores between
corresponding elements and a global consistency score among all matched
elements, while also enforcing a matching constraint between the two sets. The
SSMN significantly outperforms several strong baselines on three label transfer
scenarios: diagram-to-diagram, evaluated on a new diagram dataset of over 200
categories; image-to-image, evaluated on a dataset built on top of the Pascal
Part Dataset; and image-to-diagram, evaluated on transferring labels across
these datasets.

本文提出一种结构化的集合匹配网络，使用卷积神经网络来标记一个目标图像中的多个部分，该网络被广泛地应用于三种不同的标签传递场景中的图像匹配任务，并且明显优于几个强基线方法。