Understanding spatial location and relationships is a fundamental capability
for modern artificial intelligence systems. Insights from human spatial
cognition provide valuable guidance in this domain. Recent neuroscientific
discoveries have highlighted the role of grid cells as a fundamental neural
component for spatial representation, including distance computation, path
integration, and scale discernment. In this paper, we introduce a novel
positional encoding scheme inspired by Fourier analysis and the latest findings
in computational neuroscience regarding grid cells. Assuming that grid cells
encode spatial position through a summation of Fourier basis functions, we
demonstrate the translational invariance of the grid representation during
inner product calculations. Additionally, we derive an optimal grid scale ratio
for multi-dimensional Euclidean spaces based on principles of biological
efficiency. Utilizing these computational principles, we have developed a
**Grid**-cell inspired **Positional Encoding** technique, termed **GridPE**,
for encoding locations within high-dimensional spaces. We integrated GridPE
into the Pyramid Vision Transformer architecture. Our theoretical analysis
shows that GridPE provides a unifying framework for positional encoding in
arbitrary high-dimensional spaces. Experimental results demonstrate that GridPE
significantly enhances the performance of transformers, underscoring the
importance of incorporating neuroscientific insights into the design of
artificial intelligence systems.

通过傅立叶分析和计算神经科学的最新发现，本研究引入一种新颖的位置编码方案，受到网格细胞的启发，利用嵌入到金字塔视觉变换器架构中的 GridPE 技术，提供了一个在任意高维空间中进行位置编码的统一框架，并在转换器的性能上显著提高表现。