In recent years, video instance segmentation (VIS) has been largely advanced
by offline models, while online models gradually attracted less attention
possibly due to their inferior performance. However, online methods have their
inherent advantage in handling long video sequences and ongoing videos while
offline models fail due to the limit of computational resources. Therefore, it
would be highly desirable if online models can achieve comparable or even
better performance than offline models. By dissecting current online models and
offline models, we demonstrate that the main cause of the performance gap is
the error-prone association between frames caused by the similar appearance
among different instances in the feature space. Observing this, we propose an
online framework based on contrastive learning that is able to learn more
discriminative instance embeddings for association and fully exploit history
information for stability. Despite its simplicity, our method outperforms all
online and offline methods on three benchmarks. Specifically, we achieve 49.5
AP on YouTube-VIS 2019, a significant improvement of 13.2 AP and 2.1 AP over
the prior online and offline art, respectively. Moreover, we achieve 30.2 AP on
OVIS, a more challenging dataset with significant crowding and occlusions,
surpassing the prior art by 14.8 AP. The proposed method won first place in the
video instance segmentation track of the 4th Large-scale Video Object
Segmentation Challenge (CVPR2022). We hope the simplicity and effectiveness of
our method, as well as our insight into current methods, could shed light on
the exploration of VIS models.

通过比较现有的在线模型和离线模型，提出了一种基于对比学习的在线框架，以学习更具区分性的实例嵌入并充分利用历史信息实现关联，显著提高了视频实例分割的性能，特别是在挑战性较大的 OVIS 数据集上表现突出，赢得了第四届大规模视频对象分割挑战赛 (CVPR2022) 视频实例分割赛道的第一名。

在线模型在视频实例分割中的防御

In Defense of Online Models for Video Instance Segmentation

Feature selection can be a crucial factor in obtaining robust and accurate
predictions. Online feature selection models, however, operate under
considerable restrictions; they need to efficiently extract salient input
features based on a bounded set of observations, while enabling robust and
accurate predictions. In this work, we introduce FIRES, a novel framework for
online feature selection. The proposed feature weighting mechanism leverages
the importance information inherent in the parameters of a predictive model. By
treating model parameters as random variables, we can penalize features with
high uncertainty and thus generate more stable feature sets. Our framework is
generic in that it leaves the choice of the underlying model to the user.
Strikingly, experiments suggest that the model complexity has only a minor
effect on the discriminative power and stability of the selected feature sets.
In fact, using a simple linear model, FIRES obtains feature sets that compete
with state-of-the-art methods, while dramatically reducing computation time. In
addition, experiments show that the proposed framework is clearly superior in
terms of feature selection stability.

本文提出了一种基于 FIRES 的在线特征选择新框架，通过处理模型参数作为随机变量惩罚不确定性高的特征，选择更加稳定的特征集，并实验证明该框架在稳定性方面显著优于现有方法。