Open-set action recognition is to reject unknown human action cases which are
out of the distribution of the training set. Existing methods mainly focus on
learning better uncertainty scores but dismiss the importance of feature
representations. We find that features with richer semantic diversity can
significantly improve the open-set performance under the same uncertainty
scores. In this paper, we begin with analyzing the feature representation
behavior in the open-set action recognition (OSAR) problem based on the
information bottleneck (IB) theory, and propose to enlarge the
instance-specific (IS) and class-specific (CS) information contained in the
feature for better performance. To this end, a novel Prototypical Similarity
Learning (PSL) framework is proposed to keep the instance variance within the
same class to retain more IS information. Besides, we notice that unknown
samples sharing similar appearances to known samples are easily misclassified
as known classes. To alleviate this issue, video shuffling is further
introduced in our PSL to learn distinct temporal information between original
and shuffled samples, which we find enlarges the CS information. Extensive
experiments demonstrate that the proposed PSL can significantly boost both the
open-set and closed-set performance and achieves state-of-the-art results on
multiple benchmarks. Code is available at this https URL

本文提出了一种新的框架 —— 基于原型相似性学习（PSL），该框架通过在特征中保留实例和类的信息来提高开放集行动识别问题的性能，并引入了视频洗牌来增加类特定信息。