The Scene Representation Transformer (SRT) is a recent method to render novel views at interactive rates. Since SRT uses camera poses with respect to an arbitrarily chosen reference camera, it is not invariant to the order of the input views. As a result, SRT is not directly applicable to large-scale scenes where the reference frame would need to be changed regularly. In this work, we propose Relative Pose Attention SRT (RePAST): Instead of fixing a reference frame at the input, we inject pairwise relative camera pose information directly into the attention mechanism of the Transformers. This leads to a model that is by definition invariant to the choice of any global reference frame, while still retaining the full capabilities of the original method. Empirical results show that adding this invariance to the model does not lead to a loss in quality. We believe that this is a step towards applying fully latent transformer-based rendering methods to large-scale scenes.

本文提出了一种名为Relative Pose Attention SRT(RePAST)的方法，通过将相对的相机位姿信息注入到Transformers的注意机制中，使得其对任何全局参考帧的选择都具有不变性，同时仍保留了原始方法的全部功能，从而可以将完全潜在的基于Transformer的渲染方法应用于大规模场景。

RePAST：相对姿态注意场景表示变换器