We tackle the task of scene flow estimation from point clouds. Given a source
and a target point cloud, the objective is to estimate a translation from each
point in the source point cloud to the target, resulting in a 3D motion vector
field. Previous dominant scene flow estimation methods require complicated
coarse-to-fine or recurrent architectures as a multi-stage refinement. In
contrast, we propose a significantly simpler single-scale one-shot global
matching to address the problem. Our key finding is that reliable feature
similarity between point pairs is essential and sufficient to estimate accurate
scene flow. To this end, we propose to decompose the feature extraction step
via a hybrid local-global-cross transformer architecture which is crucial to
accurate and robust feature representations. Extensive experiments show that
GMSF sets a new state-of-the-art on multiple scene flow estimation benchmarks.
On FlyingThings3D, with the presence of occlusion points, GMSF reduces the
outlier percentage from the previous best performance of 27.4% to 11.7%. On
KITTI Scene Flow, without any fine-tuning, our proposed method shows
state-of-the-art performance.

本文提出了一种使用单尺度全局匹配的方法，使用混合的本地 - 全局 - 交叉转换器架构分解了特征提取步骤，以精确稳健的特征表示的可靠特征相似性为关键，并在多个场景流估计基准测试中取得了最新的最先进的性能。

GMSF: 全局匹配场景流

GMSF: Global Matching Scene Flow

Optical flow estimation is a fundamental task in computer vision. Recent
direct-regression methods using deep neural networks achieve remarkable
performance improvement. However, they do not explicitly capture long-term
motion correspondences and thus cannot handle large motions effectively. In
this paper, inspired by the traditional matching-optimization methods where
matching is introduced to handle large displacements before energy-based
optimizations, we introduce a simple but effective global matching step before
the direct regression and develop a learning-based matching-optimization
framework, namely GMFlowNet. In GMFlowNet, global matching is efficiently
calculated by applying argmax on 4D cost volumes. Additionally, to improve the
matching quality, we propose patch-based overlapping attention to extract large
context features. Extensive experiments demonstrate that GMFlowNet outperforms
RAFT, the most popular optimization-only method, by a large margin and achieves
state-of-the-art performance on standard benchmarks. Thanks to the matching and
overlapping attention, GMFlowNet obtains major improvements on the predictions
for textureless regions and large motions. Our code is made publicly available
at this https URL

本文提出了一种学习匹配 - 优化框架 GMFlowNet，将全局匹配引入到直接回归之前，同时引入了基于补丁的重叠注意力来提高匹配质量，实验结果表明 GMFlowNet 在标准基准上性能最佳，能够有效处理大运动和无纹理区域的挑战。

利用重叠关注全局匹配进行光流估计

Global Matching with Overlapping Attention for Optical Flow Estimation

Learning-based optical flow estimation has been dominated with the pipeline
of cost volume with convolutions for flow regression, which is inherently
limited to local correlations and thus is hard to address the long-standing
challenge of large displacements. To alleviate this, the state-of-the-art
framework RAFT gradually improves its prediction quality by using a large
number of iterative refinements, achieving remarkable performance but
introducing linearly increasing inference time. To enable both high accuracy
and efficiency, we completely revamp the dominant flow regression pipeline by
reformulating optical flow as a global matching problem, which identifies the
correspondences by directly comparing feature similarities. Specifically, we
propose a GMFlow framework, which consists of three main components: a
customized Transformer for feature enhancement, a correlation and softmax layer
for global feature matching, and a self-attention layer for flow propagation.
We further introduce a refinement step that reuses GMFlow at higher feature
resolution for residual flow prediction. Our new framework outperforms
31-refinements RAFT on the challenging Sintel benchmark, while using only one
refinement and running faster, suggesting a new paradigm for accurate and
efficient optical flow estimation. Code is available at
this https URL

通过将光流重新描述为全局匹配问题，并使用定制 Transformer 进行特征增强、相关性和 softmax 层进行全局特征匹配、以及自注意力层进行流传播来构建 GMFlow 框架，从而实现高准确性和高效率的光流估计。