Assuming the source label space subsumes the target one, Partial Video Domain
Adaptation (PVDA) is a more general and practical scenario for cross-domain
video classification problems. The key challenge of PVDA is to mitigate the
negative transfer caused by the source-only outlier classes. To tackle this
challenge, a crucial step is to aggregate target predictions to assign class
weights by up-weighing target classes and down-weighing outlier classes.
However, the incorrect predictions of class weights can mislead the network and
lead to negative transfer. Previous works improve the class weight accuracy by
utilizing temporal features and attention mechanisms, but these methods may
fall short when trying to generate accurate class weight when domain shifts are
significant, as in most real-world scenarios. To deal with these challenges, we
propose the Multi-modality Cluster-calibrated partial Adversarial Network
(MCAN). MCAN enhances video feature extraction with multi-modal features from
multiple temporal scales to form more robust overall features. It utilizes a
novel class weight calibration method to alleviate the negative transfer caused
by incorrect class weights. The calibration method tries to identify and weigh
correct and incorrect predictions using distributional information implied by
unsupervised clustering. Extensive experiments are conducted on prevailing PVDA
benchmarks, and the proposed MCAN achieves significant improvements when
compared to state-of-the-art PVDA methods.

本文介绍了一种名为 MCAN 的新型神经网络模型，采用多模态特征结合不同时间尺度的视频特征提取，利用聚类方法的分布信息减轻了错误类权重带来的负面影响，该模型已在 PVDA 基准测试中得到了显著的提高。