We present a new test-time optimization method for estimating dense and
long-range motion from a video sequence. Prior optical flow or particle video
tracking algorithms typically operate within limited temporal windows,
struggling to track through occlusions and maintain global consistency of
estimated motion trajectories. We propose a complete and globally consistent
motion representation, dubbed OmniMotion, that allows for accurate, full-length
motion estimation of every pixel in a video. OmniMotion represents a video
using a quasi-3D canonical volume and performs pixel-wise tracking via
bijections between local and canonical space. This representation allows us to
ensure global consistency, track through occlusions, and model any combination
of camera and object motion. Extensive evaluations on the TAP-Vid benchmark and
real-world footage show that our approach outperforms prior state-of-the-art
methods by a large margin both quantitatively and qualitatively. See our
project page for more results: this http URL

我们提出了一种新的测试时间优化方法，用于从视频序列中估计密集和长程运动。该方法使用全局一致的运动表示法，并在局部和规范空间之间使用双向对应实现像素级跟踪，能够确保全局一致性，跟踪通过遮挡的像素，并模拟相机和物体运动的任意组合。