We present a new generalizable NeRF method that is able to directly
generalize to new unseen scenarios and perform novel view synthesis with as few
as two source views. The key to our approach lies in the explicitly modeled
correspondence matching information, so as to provide the geometry prior to the
prediction of NeRF color and density for volume rendering. The explicit
correspondence matching is quantified with the cosine similarity between image
features sampled at the 2D projections of a 3D point on different views, which
is able to provide reliable cues about the surface geometry. Unlike previous
methods where image features are extracted independently for each view, we
consider modeling the cross-view interactions via Transformer cross-attention,
which greatly improves the feature matching quality. Our method achieves
state-of-the-art results on different evaluation settings, with the experiments
showing a strong correlation between our learned cosine feature similarity and
volume density, demonstrating the effectiveness and superiority of our proposed
method. Code is at this https URL

该论文提出了一种新的 NeRF 方法，具有可推广性，能够利用极少的两个来源视图直接推广到新的看不见的场景并进行新的视图合成。该方法的关键在于显式建模的对应匹配信息，以在 NeRF 颜色和密度的预测之前提供几何先验，从而实现体渲染。该方法通过 Transformer 交叉关注模型跨视图的交互作用来建模，大大提高了特征匹配的质量，并实现了在不同评估设置中的最新成果。实验表明了该方法学习到的余弦特征相似性和体密度之间的强相关性，证明了该方法的有效性和优越性。