In this paper we propose the Ray-Patch decoder, a novel model to efficiently
query transformers to decode implicit representations into target views. Our
Ray-Patch decoding reduces the computational footprint up to two orders of
magnitude compared to previous models, without losing global attention, and
hence maintaining specific task metrics. The key idea of our novel decoder is
to split the target image into a set of patches, then querying the transformer
for each patch to extract a set of feature vectors, which are finally decoded
into the target image using convolutional layers. Our experimental results
quantify the effectiveness of our method, specifically the notable boost in
rendering speed and equal specific task metrics for different baselines and
datasets.

本文提出了一种新的 Ray-Patch 解码器模型，可以有效地从 implicit representations 中解码出目标视图，将目标图像分成一组 patch，然后对于每个 patch 查询 transformer 以提取一组特征向量，最后使用卷积层将其解码为目标图像，相比以前的模型，我们的 Ray-Patch 解码可以将计算量减少两个数量级，同时保持全局注意力且维持特定任务指标。实验结果表明，我们的方法具有显著的渲染速度提升和相等的特定任务指标，适用于不同的基线和数据集。