In recent years, deep-learning-based visual object trackers have been studied
thoroughly, but handling occlusions and/or rapid motion of the target remains
challenging. In this work, we argue that conditioning on the natural language
(NL) description of a target provides information for longer-term invariance,
and thus helps cope with typical tracking challenges. However, deriving a
formulation to combine the strengths of appearance-based tracking with the
language modality is not straightforward. We propose a novel deep
tracking-by-detection formulation that can take advantage of NL descriptions.
Regions that are related to the given NL description are generated by a
proposal network during the detection phase of the tracker. Our LSTM based
tracker then predicts the update of the target from regions proposed by the NL
based detection phase. In benchmarks, our method is competitive with state of
the art trackers, while it outperforms all other trackers on targets with
unambiguous and precise language annotations. It also beats the
state-of-the-art NL tracker when initializing without a bounding box. Our
method runs at over 30 fps on a single GPU.

该研究提出一种利用自然语言描述的深度追踪器，通过使用 LSTM 跟踪器预测目标的更新，可以以每秒超过 30 帧的速度处理具有模糊语言注释的目标，展示出色的性能。