Modeling global geometric context while maintaining equivariance is crucial
for accurate predictions in many fields such as biology, chemistry, or vision.
Yet, this is challenging due to the computational demands of processing
high-dimensional data at scale. Existing approaches such as equivariant
self-attention or distance-based message passing, suffer from quadratic
complexity with respect to sequence length, while localized methods sacrifice
global information. Inspired by the recent success of state-space and
long-convolutional models, in this work, we introduce SE(3)-Hyena operator, an
equivariant long-convolutional model based on the Hyena operator. The
SE(3)-Hyena captures global geometric context at sub-quadratic complexity while
maintaining equivariance to rotations and translations. Evaluated on
equivariant associative recall and n-body modeling, SE(3)-Hyena matches or
outperforms equivariant self-attention while requiring significantly less
memory and computational resources for long sequences. Our model processes the
geometric context of 20k tokens x3.5 times faster than the equivariant
transformer and allows x175 longer a context within the same memory budget.

基于 Hyena 操作符的等变长卷积模型 SE (3)-Hyena，在保持旋转和平移等变性的同时，以次二次复杂度捕捉全局几何上下文，处理长序列时比等变自注意力机制更省内存和计算资源。