The integration of knowledge extracted from diverse models, whether described
by domain experts or generated by machine learning algorithms, has historically
been challenged by the absence of a suitable framework for specifying and
integrating structures, learning processes, data transformations, and data
models or rules. In this work, we extend algebraic specification methods to
address these challenges within such a framework.
In our work, we tackle the challenging task of developing a comprehensive
framework for defining and analyzing deep learning architectures. We believe
that previous efforts have fallen short by failing to establish a clear
connection between the constraints a model must adhere to and its actual
implementation.
Our methodology employs graphical structures that resemble Ehresmann's
sketches, interpreted within a universe of fuzzy sets. This approach offers a
unified theory that elegantly encompasses both deterministic and
non-deterministic neural network designs. Furthermore, we highlight how this
theory naturally incorporates fundamental concepts from computer science and
automata theory. Our extended algebraic specification framework, grounded in
graphical structures akin to Ehresmann's sketches, offers a promising solution
for integrating knowledge across disparate models and domains. By bridging the
gap between domain-specific expertise and machine-generated insights, we pave
the way for more comprehensive, collaborative, and effective approaches to
knowledge integration and modeling.

我们扩展了代数规范方法以解决从不同模型中提取的知识的集成问题，并且通过将图形结构与 Ehresmann 的素描结构相结合，提出了一个统一的理论，它能够优雅地包含确定性和非确定性神经网络设计的基本概念。这个拓展的代数规范框架为跨不同模型和领域的知识集成提供了一个有希望的解决方案。

范畴符号学：知识整合基础

Categorical semiotics: Foundations for Knowledge Integration

We introduce a novel positional encoding strategy for Transformer-style
models, addressing the shortcomings of existing, often ad hoc, approaches. Our
framework provides a flexible mapping from the algebraic specification of a
domain to an interpretation as orthogonal operators. This design preserves the
algebraic characteristics of the source domain, ensuring that the model upholds
the desired structural properties. Our scheme can accommodate various
structures, including sequences, grids and trees, as well as their
compositions. We conduct a series of experiments to demonstrate the practical
applicability of our approach. Results suggest performance on par with or
surpassing the current state-of-the-art, without hyperparameter optimizations
or ``task search'' of any kind. Code will be made available at
\url{github.com/konstantinosKokos/UnitaryPE}.

我们介绍了一种新颖的位置编码策略，用于 Transformer 风格模型，解决了现有方法的不足之处。我们的框架提供了一种灵活的映射，从一个领域的代数规范到正交操作符的解释。该设计保留了源域的代数特性，确保模型满足所需的结构属性。我们的方案可以适应各种结构，包括序列、网格和树形结构，以及它们的组合。我们进行了一系列实验证明我们方法的实际适用性。结果表明，在没有超参数优化或任何 “任务搜索” 的情况下，性能可以达到或超过当前的最新水平。代码将在 github.com/konstantinosKokos/UnitaryPE 上提供。