We introduce a category-theoretic diagrammatic formalism in order to
systematically relate and reason about machine learning models. Our diagrams
present architectures intuitively but without loss of essential detail, where
natural relationships between models are captured by graphical transformations,
and important differences and similarities can be identified at a glance. In
this paper, we focus on attention mechanisms: translating folklore into
mathematical derivations, and constructing a taxonomy of attention variants in
the literature. As a first example of an empirical investigation underpinned by
our formalism, we identify recurring anatomical components of attention, which
we exhaustively recombine to explore a space of variations on the attention
mechanism.

我们引入一种范畴论的图示形式，以系统性地关联和推理机器学习模型。我们的图示以直观但不丢失关键细节的方式呈现架构，其中模型之间的自然关系通过图形变换捕捉，并且重要的差异和相似之处一目了然。在本文中，我们着重于注意力机制：将民间传说转化为数学推导，并在文献中构建了一个注意力变体的分类学。作为我们形式主义支持下的经验研究的第一个例子，我们识别了注意力的重复解剖组成部分，我们全面重组它们以探索注意力机制的变化空间。

注意力的解剖学研究

On the Anatomy of Attention

Motifs are distinctive recurring elements found in folklore that have
significance as communicative devices in news, literature, press releases, and
propaganda. Motifs concisely imply a large constellation of culturally-relevant
information, and their broad usage suggests their cognitive importance as
touchstones of cultural knowledge, making their detection a worthy step toward
culturally-aware natural language processing tasks. Until now, folklorists and
others interested in motifs have only extracted motifs from narratives
manually. We present a preliminary report on the development of a system for
automatically detecting motifs. We briefly describe an annotation effort to
produce data for training motif detection, which is on-going. We describe our
in-progress architecture in detail, which aims to capture, in part, how people
determine whether or not a motif candidate is being used in a motific way. This
description includes a test of an off-the-shelf metaphor detector as a feature
for motif detection, which achieves a F1 of 0.35 on motifs and a macro-average
F1 of 0.21 across four categories which we assign to motif candidates.

本文初步报告了一项关于自动检测民间传说中的 Motifs 的系统开发，其中包括用于训练的标记工作、在建构架构中设计用于捕捉 Motifs 的关键特征、和测试案例的测量结果，进一步证明 Motifs 的重要性以及其在文化认知领域自动化处理的潜在价值。

在桥底下寻找巨魔：一个图像模式检测器的初步工作

Finding Trolls Under Bridges: Preliminary Work on a Motif Detector

Folklore, a solid branch of folk literature, is the hallmark of any nation or
any society. Such as oral tradition; as proverbs or jokes, it also includes
material culture as well as traditional folk beliefs, and various customs.
Bengali folklore is as rich in-depth as it is amazing. Nevertheless, in the
womb of time, it is determined to sustain its existence. Therefore, our aim in
this study is to make our rich folklore more comprehensible to everyone in a
more sophisticated computational way. Some studies concluded various aspects of
the Bengali language with NLP. Our proposed model is to be specific for Bengali
folklore. Technically, it will be the first step towards Bengali natural
language processing for studying and analyzing the folklore of Bengal.

本文旨在将孟加拉民间故事的丰富性以更精密的计算方式呈现给每个人。本研究中提出的模型是研究和分析孟加拉民间故事时，孟加拉语自然语言处理的第一步。