Video object detection is challenging in the presence of appearance
deterioration in certain video frames. Therefore, it is a natural choice to
aggregate temporal information from other frames of the same video into the
current frame. However, RoI Align, as one of the most core procedures of video
detectors, still remains extracting features from a single-frame feature map
for proposals, making the extracted RoI features lack temporal information from
videos. In this work, considering the features of the same object instance are
highly similar among frames in a video, a novel Temporal RoI Align operator is
proposed to extract features from other frames feature maps for current frame
proposals by utilizing feature similarity. The proposed Temporal RoI Align
operator can extract temporal information from the entire video for proposals.
We integrate it into single-frame video detectors and other state-of-the-art
video detectors, and conduct quantitative experiments to demonstrate that the
proposed Temporal RoI Align operator can consistently and significantly boost
the performance. Besides, the proposed Temporal RoI Align can also be applied
into video instance segmentation. Codes are available at
this https URL

在视频物体检测中，使用提取视频中当前帧之前的帧中的信息对当前帧进行检测以提取时序信息是一种自然的选择，本文提出了一种基于特征相似性的新型 Temporal RoI Align 算子，用于从当前视频中的其它帧特征图中提取与当前帧相似的结果，能够提取整个视频的时序信息，并将其整合到单帧视频检测器和其他现有视频检测器中，经定量实验表明，所提出的 Temporal RoI Align 算子可以显著提高检测性能。