In this paper, we present TransMVSNet, based on our exploration of feature
matching in multi-view stereo (MVS). We analogize MVS back to its nature of a
feature matching task and therefore propose a powerful Feature Matching
Transformer (FMT) to leverage intra- (self-) and inter- (cross-) attention to
aggregate long-range context information within and across images. To
facilitate a better adaptation of the FMT, we leverage an Adaptive Receptive
Field (ARF) module to ensure a smooth transit in scopes of features and bridge
different stages with a feature pathway to pass transformed features and
gradients across different scales. In addition, we apply pair-wise feature
correlation to measure similarity between features, and adopt
ambiguity-reducing focal loss to strengthen the supervision. To the best of our
knowledge, TransMVSNet is the first attempt to leverage Transformer into the
task of MVS. As a result, our method achieves state-of-the-art performance on
DTU dataset, Tanks and Temples benchmark, and BlendedMVS dataset. The code of
our method will be made available at this https URL .

本研究提出了 TransMVSNet，它是基于多视图立体视觉（MVS）中的特征匹配的探索而来的，利用 Feature Matching Transformer 实现自内、交叉注意力聚合图像内外的长程上下文信息来完成 MVS 任务，并借助 Adaptive Receptive Field 模块和 Pair-wise feature correlation 量化特征的相似性和优化任务效果。实验结果表明，该方法在多个基准数据集上取得了最优的性能。