Combining the Color and Event cameras (also called Dynamic Vision Sensors,
DVS) for robust object tracking is a newly emerging research topic in recent
years. Existing color-event tracking framework usually contains multiple
scattered modules which may lead to low efficiency and high computational
complexity, including feature extraction, fusion, matching, interactive
learning, etc. In this paper, we propose a single-stage backbone network for
Color-Event Unified Tracking (CEUTrack), which achieves the above functions
simultaneously. Given the event points and RGB frames, we first transform the
points into voxels and crop the template and search regions for both
modalities, respectively. Then, these regions are projected into tokens and
parallelly fed into the unified Transformer backbone network. The output
features will be fed into a tracking head for target object localization. Our
proposed CEUTrack is simple, effective, and efficient, which achieves over 75
FPS and new SOTA performance. To better validate the effectiveness of our model
and address the data deficiency of this task, we also propose a generic and
large-scale benchmark dataset for color-event tracking, termed COESOT, which
contains 90 categories and 1354 video sequences. Additionally, a new evaluation
metric named BOC is proposed in our evaluation toolkit to evaluate the
prominence with respect to the baseline methods. We hope the newly proposed
method, dataset, and evaluation metric provide a better platform for
color-event-based tracking. The dataset, toolkit, and source code will be
released on: https://github.com/Event-AHU/COESOT.

本文提出了一种名为 CEUTrack 的单阶段骨干网络，该网络通过 Transformer 骨干网络实现了特征提取、融合、匹配和互动学习等功能，用于对基于颜色和事件的物体进行跟踪。此外，我们还提出了一个大规模基准数据集 COESOT，并提出了一种新的评估指标 BOC。