Omnidirectional stereo matching (OSM) is an essential and reliable means for
$360^{\circ}$ depth sensing. However, following earlier works on conventional
stereo matching, prior state-of-the-art (SOTA) methods rely on a 3D
encoder-decoder block to regularize the cost volume, causing the whole system
complicated and sub-optimal results. Recently, the Recurrent All-pairs Field
Transforms (RAFT) based approach employs the recurrent update in 2D and has
efficiently improved image-matching tasks, \ie, optical flow, and stereo
matching. To bridge the gap between OSM and RAFT, we mainly propose an opposite
adaptive weighting scheme to seamlessly transform the outputs of spherical
sweeping of OSM into the required inputs for the recurrent update, thus
creating a recurrent omnidirectional stereo matching (RomniStereo) algorithm.
Furthermore, we introduce two techniques, \ie, grid embedding and adaptive
context feature generation, which also contribute to RomniStereo's performance.
Our best model improves the average MAE metric by 40.7\% over the previous SOTA
baseline across five datasets. When visualizing the results, our models
demonstrate clear advantages on both synthetic and realistic examples. The code
is available at https://github.com/HalleyJiang/RomniStereo.

提出了一种名为 RomniStereo 的递归全向立体匹配算法，通过对全向立体匹配的输出进行权重调整，将其转化为递归更新的输入，同时引入了格点嵌入和自适应上下文特征生成等技术，实现在多个数据集上平均 MAE 度量指标提升了 40.7%，并在合成和真实示例上展现明显的优势。

RomniStereo：循环全方位立体匹配

RomniStereo: Recurrent Omnidirectional Stereo Matching

In this paper, we present a novel deep-learning model for deformable
registration of ultrasound images and an unsupervised approach to training this
model. Our network employs recurrent all-pairs field transforms (RAFT) and a
spatial transformer network (STN) to generate displacement fields at online
rates (apprx. 30 Hz) and accurately track pixel movement. We call our approach
unsupervised recurrent all-pairs field transforms (U-RAFT). In this work, we
use U-RAFT to track pixels in a sequence of ultrasound images to cancel out
respiratory motion in lung ultrasound images. We demonstrate our method on
in-vivo porcine lung videos. We show a reduction of 76% in average pixel
movement in the porcine dataset using respiratory motion compensation strategy.
We believe U-RAFT is a promising tool for compensating different kinds of
motions like respiration and heartbeat in ultrasound images of deformable
tissue.

本文提出了一种基于深度学习的新型超声图像变形配准模型及一种无监督的训练方法，该模型使用循环全对场变换和空间变换网络来在在线速率下（约 30 Hz）生成位移场并精确地跟踪像素移动，采用该方法来在活体猪肺视频中跟踪像素，演示了呼吸运动补偿策略，减少了猪数据集中平均像素运动的 76％，我们相信 U-RAFT 是一个有前途的工具，可用于补偿可变形组织的超声图像中的不同运动，如呼吸和心跳。

无监督可变形图像配准用于超声图像呼吸运动补偿

Unsupervised Deformable Image Registration for Respiratory Motion  Compensation in Ultrasound Images

We introduce Recurrent All-Pairs Field Transforms (RAFT), a new deep network
architecture for optical flow. RAFT extracts per-pixel features, builds
multi-scale 4D correlation volumes for all pairs of pixels, and iteratively
updates a flow field through a recurrent unit that performs lookups on the
correlation volumes. RAFT achieves state-of-the-art performance. On KITTI, RAFT
achieves an F1-all error of 5.10%, a 16% error reduction from the best
published result (6.10%). On Sintel (final pass), RAFT obtains an
end-point-error of 2.855 pixels, a 30% error reduction from the best published
result (4.098 pixels). In addition, RAFT has strong cross-dataset
generalization as well as high efficiency in inference time, training speed,
and parameter count. Code is available at this https URL

RAFT 是一个新的深度网络结构，用于光流问题，它提取每个像素的特征，为所有像素建立多尺度 4D 相关性体，并通过一个循环单元，通过对相关性体的查找，迭代更新光流场。RAFT 在 KITTI 和 Sintel 数据集上实现了最新颖的性能，并且具有高效率的推断时间、训练速度和参数计数。