In 3D Visual Question Answering (3D VQA), the scarcity of fully annotated
data and limited visual content diversity hampers the generalization to novel
scenes and 3D concepts (e.g., only around 800 scenes are utilized in ScanQA and
SQA dataset). Current approaches resort supplement 3D reasoning with 2D
information. However, these methods face challenges: either they use top-down
2D views that introduce overly complex and sometimes question-irrelevant visual
clues, or they rely on globally aggregated scene/image-level representations
from 2D VLMs, losing the fine-grained vision-language correlations. To overcome
these limitations, our approach utilizes question-conditional 2D view selection
procedure, pinpointing semantically relevant 2D inputs for crucial visual
clues. We then integrate this 2D knowledge into the 3D-VQA system via a
two-branch Transformer structure. This structure, featuring a Twin-Transformer
design, compactly combines 2D and 3D modalities and captures fine-grained
correlations between modalities, allowing them mutually augmenting each other.
Integrating proposed mechanisms above, we present BridgeQA, that offers a fresh
perspective on multi-modal transformer-based architectures for 3D-VQA.
Experiments validate that BridgeQA achieves state-of-the-art on 3D-VQA datasets
and significantly outperforms existing solutions. Code is available at
$\href{this https URL}{\text{this URL}}$.

利用问题条件的 2D 视图选择过程，将 2D 知识与 3D-VQA 系统进行整合，通过双 Transformer 结构紧密结合 2D 和 3D 模态，并捕捉模态之间的细粒度相关性，实现了面向 3D-VQA 的多模态基于 Transformer 的架构。