Open-set action recognition is to reject unknown human action cases which are
out of the distribution of the training set. Existing methods mainly focus on
learning better uncertainty scores but dismiss the importance of feature
representations. We find that features with richer semantic diversity can
significantly improve the open-set performance under the same uncertainty
scores. In this paper, we begin with analyzing the feature representation
behavior in the open-set action recognition (OSAR) problem based on the
information bottleneck (IB) theory, and propose to enlarge the
instance-specific (IS) and class-specific (CS) information contained in the
feature for better performance. To this end, a novel Prototypical Similarity
Learning (PSL) framework is proposed to keep the instance variance within the
same class to retain more IS information. Besides, we notice that unknown
samples sharing similar appearances to known samples are easily misclassified
as known classes. To alleviate this issue, video shuffling is further
introduced in our PSL to learn distinct temporal information between original
and shuffled samples, which we find enlarges the CS information. Extensive
experiments demonstrate that the proposed PSL can significantly boost both the
open-set and closed-set performance and achieves state-of-the-art results on
multiple benchmarks. Code is available at this https URL

本文提出了一种新的框架 —— 基于原型相似性学习（PSL），该框架通过在特征中保留实例和类的信息来提高开放集行动识别问题的性能，并引入了视频洗牌来增加类特定信息。

扩展实例特定和类别特定信息以进行开放式动作识别

Enlarging Instance-specific and Class-specific Information for Open-set  Action Recognition

We propose a new technique, Singular Vector Canonical Correlation Analysis
(SVCCA), a tool for quickly comparing two representations in a way that is both
invariant to affine transform (allowing comparison between different layers and
networks) and fast to compute (allowing more comparisons to be calculated than
with previous methods). We deploy this tool to measure the intrinsic
dimensionality of layers, showing in some cases needless over-parameterization;
to probe learning dynamics throughout training, finding that networks converge
to final representations from the bottom up; to show where class-specific
information in networks is formed; and to suggest new training regimes that
simultaneously save computation and overfit less. Code:
this https URL

提出了 Singular Vector Canonical Correlation Analysis（SVCCA）技术，用于快速比较两个表示方式，并测量网络层的内在维度、学习动态、类特定信息和建议新的训练方案。