We present a novel model for Tracking Any Point (TAP) that effectively tracks
any queried point on any physical surface throughout a video sequence. Our
approach employs two stages: (1) a matching stage, which independently locates
a suitable candidate point match for the query point on every other frame, and
(2) a refinement stage, which updates both the trajectory and query features
based on local correlations. The resulting model surpasses all baseline methods
by a significant margin on the TAP-Vid benchmark, as demonstrated by an
approximate 20% absolute average Jaccard (AJ) improvement on DAVIS. Our model
facilitates fast inference on long and high-resolution video sequences. On a
modern GPU, our implementation has the capacity to track points faster than
real-time. Visualizations, source code, and pretrained models can be found on
our project webpage.

本研究提出了一种名为 TAP 的新型模型，能在视频序列中高效地跟踪任何查询点，包含匹配阶段和细化阶段，且比基准方法表现更优，可快速适用于长且高分辨率的视频序列，具有实时的跟踪速度。

TAPIR: 使用逐帧初始化和时间细化技术追踪任意点

TAPIR: Tracking Any Point with per-frame Initialization and temporal  Refinement

Dense feature matching is an important computer vision task that involves
estimating all correspondences between two images of a 3D scene. In this paper,
we revisit robust losses for matching from a Markov chain perspective, yielding
theoretical insights and large gains in performance. We begin by constructing a
unifying formulation of matching as a Markov chain, based on which we identify
two key stages which we argue should be decoupled for matching. The first is
the coarse stage, where the estimated result needs to be globally consistent.
The second is the refinement stage, where the model needs precise localization
capabilities. Inspired by the insight that these stages concern distinct
issues, we propose a coarse matcher following the regression-by-classification
paradigm that provides excellent globally consistent, albeit not exactly
localized, matches. This is followed by a local feature refinement stage using
well-motivated robust regression losses, yielding extremely precise matches.
Our proposed approach, which we call RoMa, achieves significant improvements
compared to the state-of-the-art. Code is available at
this https URL

从马尔可夫链的角度重新审视强健损失，将匹配问题分解为粗匹配和细匹配两个阶段，提出了一种 RoMa 算法以进行本地特征精炼，实现了显著的性能提升。