Transformers have established themselves as the leading neural network model
in natural language processing and are increasingly foundational in various
domains. In vision, the MLP-Mixer model has demonstrated competitive
performance, suggesting that attention mechanisms might not be indispensable.
Inspired by this, recent research has explored replacing attention modules with
other mechanisms, including those described by MetaFormers. However, the
theoretical framework for these models remains underdeveloped. This paper
proposes a novel perspective by integrating Krotov's hierarchical associative
memory with MetaFormers, enabling a comprehensive representation of the entire
Transformer block, encompassing token-/channel-mixing modules, layer
normalization, and skip connections, as a single Hopfield network. This
approach yields a parallelized MLP-Mixer derived from a three-layer Hopfield
network, which naturally incorporates symmetric token-/channel-mixing modules
and layer normalization. Empirical studies reveal that symmetric interaction
matrices in the model hinder performance in image recognition tasks.
Introducing symmetry-breaking effects transitions the performance of the
symmetric parallelized MLP-Mixer to that of the vanilla MLP-Mixer. This
indicates that during standard training, weight matrices of the vanilla
MLP-Mixer spontaneously acquire a symmetry-breaking configuration, enhancing
their effectiveness. These findings offer insights into the intrinsic
properties of Transformers and MLP-Mixers and their theoretical underpinnings,
providing a robust framework for future model design and optimization.

通过将 Krotov 的分层关联记忆与 MetaFormers 相结合，该论文提出了一种新的视角，将整个 Transformer 块的完整表示，包括标记 -/ 通道混合模块、层归一化和跳跃连接，作为一个单一的 Hopfield 网络。该方法产生了一个并行化的从三层 Hopfield 网络推导出的 MLP-Mixer，自然地融合了对称的标记 -/ 通道混合模块和层归一化。实证研究揭示了模型中的对称交互矩阵阻碍了图像识别任务的性能。引入破坏对称效果将对称并行化的 MLP-Mixer 的性能过渡到普通 MLP-Mixer 的性能。这表明在标准训练过程中，普通 MLP-Mixer 的权重矩阵自发地获得对称破缺配置，增强了其有效性。这些发现为 Transformer 和 MLP-Mixer 的内在特性及其理论基础提供了见解，为未来模型设计和优化提供了一个稳健的框架。

分层联想记忆、并行化 MLP-Mixer 和对称性破坏

Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry  Breaking

A packing $k$-coloring is a natural variation on the standard notion of graph
$k$-coloring, where vertices are assigned numbers from $\{1, \ldots, k\}$, and
any two vertices assigned a common color $c \in \{1, \ldots, k\}$ need to be at
a distance greater than $c$ (as opposed to $1$, in standard graph colorings).
Despite a sequence of incremental work, determining the packing chromatic
number of the infinite square grid has remained an open problem since its
introduction in 2002. We culminate the search by proving this number to be 15.
We achieve this result by improving the best-known method for this problem by
roughly two orders of magnitude. The most important technique to boost
performance is a novel, surprisingly effective propositional encoding for
packing colorings. Additionally, we developed an alternative symmetry-breaking
method. Since both new techniques are more complex than existing techniques for
this problem, a verified approach is required to trust them. We include both
techniques in a proof of unsatisfiability, reducing the trusted core to the
correctness of the direct encoding.

本文研究了 packing coloring 在图论中的应用，提出了一种新的 propositional encoding 方法加速计算，最终证明了无限正方形网格的 chromatic number 为 15。

无限方格的填色染色数为 15

The Packing Chromatic Number of the Infinite Square Grid is 15

These lecture notes introduce quantum spin systems and several computational
methods for studying their ground-state and finite-temperature properties.
Symmetry-breaking and critical phenomena are first discussed in the simpler
setting of Monte Carlo studies of classical spin systems, to illustrate
finite-size scaling at continuous and first-order phase transitions. Exact
diagonalization and quantum Monte Carlo (stochastic series expansion)
algorithms and their computer implementations are then discussed in detail.
Applications of the methods are illustrated by results for some of the most
essential models in quantum magnetism, such as the S=1/2 Heisenberg
antiferromagnet in one and two dimensions, as well as extended models useful
for studying quantum phase transitions between antiferromagnetic and
magnetically disordered states.

这篇文章详细介绍了研究量子自旋系统地面态和有限温度性质的几种计算方法，包括对称性破缺、临界现象和应用于量子磁学中最重要模型的算法。

量子自旋系统的计算研究

Computational Studies of Quantum Spin Systems

In the context of Answer Set Programming, this paper investigates
symmetry-breaking to eliminate symmetric parts of the search space and,
thereby, simplify the solution process. We propose a reduction of disjunctive
logic programs to a coloured digraph such that permutational symmetries can be
constructed from graph automorphisms. Symmetries are then broken by introducing
symmetry-breaking constraints. For this purpose, we formulate a preprocessor
that integrates a graph automorphism system. Experiments demonstrate its
computational impact.

本篇研究在回答集规划（Answer Set Programming）背景下，探讨通过对称性破除来简化解决方案的过程，提出将不相干的逻辑程序转化为有色图的缩减方法，并通过图自同构构建置换对称性，然后通过引入对称性破除约束来破坏对称性。针对此目的，我们设计了一个包含图自同构系统的预处理器。实验证明该系统具有较高的计算效率。