As transformers are equivariant to the permutation of input tokens, encoding
the positional information of tokens is necessary for many tasks. However,
since existing positional encoding schemes have been initially designed for NLP
tasks, their suitability for vision tasks, which typically exhibit different
structural properties in their data, is questionable. We argue that existing
positional encoding schemes are suboptimal for 3D vision tasks, as they do not
respect their underlying 3D geometric structure. Based on this hypothesis, we
propose a geometry-aware attention mechanism that encodes the geometric
structure of tokens as relative transformation determined by the geometric
relationship between queries and key-value pairs. By evaluating on multiple
novel view synthesis (NVS) datasets in the sparse wide-baseline multi-view
setting, we show that our attention, called Geometric Transform Attention
(GTA), improves learning efficiency and performance of state-of-the-art
transformer-based NVS models without any additional learned parameters and only
minor computational overhead.

通过提出一种几何感知注意机制 (Geometric Transform Attention, GTA) 来对几何结构进行编码，改进了基于 Transformer 的多视角合成模型的学习效率和性能，无需额外学习参数且计算开销较小。