The Audio-Visual Question Answering (AVQA) task holds significant potential
for applications. Compared to traditional unimodal approaches, the multi-modal
input of AVQA makes feature extraction and fusion processes more challenging.
Euclidean space is difficult to effectively represent multi-dimensional
relationships of data. Especially when extracting and processing data with a
tree structure or hierarchical structure, Euclidean space is not suitable as an
embedding space. Additionally, the self-attention mechanism in Transformers is
effective in capturing the dynamic relationships between elements in a
sequence. However, the self-attention mechanism's limitations in window
modeling and quadratic computational complexity reduce its effectiveness in
modeling long sequences. To address these limitations, we propose SHMamba:
Structured Hyperbolic State Space Model to integrate the advantages of
hyperbolic geometry and state space models. Specifically, SHMamba leverages the
intrinsic properties of hyperbolic space to represent hierarchical structures
and complex relationships in audio-visual data. Meanwhile, the state space
model captures dynamic changes over time by globally modeling the entire
sequence. Furthermore, we introduce an adaptive curvature hyperbolic alignment
module and a cross fusion block to enhance the understanding of hierarchical
structures and the dynamic exchange of cross-modal information, respectively.
Extensive experiments demonstrate that SHMamba outperforms previous methods
with fewer parameters and computational costs. Our learnable parameters are
reduced by 78.12\%, while the average performance improves by 2.53\%.
Experiments show that our method demonstrates superiority among all current
major methods and is more suitable for practical application scenarios.

提出了 SHMamba：结构化双曲状态空间模型，利用双曲几何和状态空间模型的优势，表示音频 - 视觉数据的分层结构和复杂关系。通过引入自适应曲率双曲对齐模块和交叉融合块，增强分层结构的理解和跨模态信息的动态交流。广泛实验证明，SHMamba 在参数和计算成本上优于以前的方法，参数减少 78.12％，平均性能提高 2.53％。实验证明我们的方法在当前主要方法中具有优势，并更适合实际应用场景。