Deep learning models have a risk of utilizing spurious clues to make
predictions, such as recognizing actions based on the background scene. This
issue can severely degrade the open-set action recognition performance when the
testing samples have different scene distributions from the training samples.
To mitigate this problem, we propose a novel method, called Scene-debiasing
Open-set Action Recognition (SOAR), which features an adversarial scene
reconstruction module and an adaptive adversarial scene classification module.
The former prevents the decoder from reconstructing the video background given
video features, and thus helps reduce the background information in feature
learning. The latter aims to confuse scene type classification given video
features, with a specific emphasis on the action foreground, and helps to learn
scene-invariant information. In addition, we design an experiment to quantify
the scene bias. The results indicate that the current open-set action
recognizers are biased toward the scene, and our proposed SOAR method better
mitigates such bias. Furthermore, our extensive experiments demonstrate that
our method outperforms state-of-the-art methods, and the ablation studies
confirm the effectiveness of our proposed modules.

通过对深度学习模型中可能存在的场景偏置进行建模和抑制，本文提出了一种新方法 —— 场景去偏置开集动作识别（SOAR），其中包括对抗场景重构模块和自适应对抗场景分类模块，实验证明该方法可以有效减轻场景偏置并提高动作识别性能。

SOAR：场景去偏的开放式行为识别

SOAR: Scene-debiasing Open-set Action Recognition

Open-set action recognition is to reject unknown human action cases which are
out of the distribution of the training set. Existing methods mainly focus on
learning better uncertainty scores but dismiss the importance of feature
representations. We find that features with richer semantic diversity can
significantly improve the open-set performance under the same uncertainty
scores. In this paper, we begin with analyzing the feature representation
behavior in the open-set action recognition (OSAR) problem based on the
information bottleneck (IB) theory, and propose to enlarge the
instance-specific (IS) and class-specific (CS) information contained in the
feature for better performance. To this end, a novel Prototypical Similarity
Learning (PSL) framework is proposed to keep the instance variance within the
same class to retain more IS information. Besides, we notice that unknown
samples sharing similar appearances to known samples are easily misclassified
as known classes. To alleviate this issue, video shuffling is further
introduced in our PSL to learn distinct temporal information between original
and shuffled samples, which we find enlarges the CS information. Extensive
experiments demonstrate that the proposed PSL can significantly boost both the
open-set and closed-set performance and achieves state-of-the-art results on
multiple benchmarks. Code is available at this https URL

本文提出了一种新的框架 —— 基于原型相似性学习（PSL），该框架通过在特征中保留实例和类的信息来提高开放集行动识别问题的性能，并引入了视频洗牌来增加类特定信息。

扩展实例特定和类别特定信息以进行开放式动作识别

Enlarging Instance-specific and Class-specific Information for Open-set  Action Recognition

Existing methods for open-set action recognition focus on novelty detection
that assumes video clips show a single action, which is unrealistic in the real
world. We propose a new method for open set action recognition and novelty
detection via MUlti-Label Evidential learning (MULE), that goes beyond previous
novel action detection methods by addressing the more general problems of
single or multiple actors in the same scene, with simultaneous action(s) by any
actor. Our Beta Evidential Neural Network estimates multi-action uncertainty
with Beta densities based on actor-context-object relation representations. An
evidence debiasing constraint is added to the objective function for
optimization to reduce the static bias of video representations, which can
incorrectly correlate predictions and static cues. We develop a learning
algorithm based on a primal-dual average scheme update to optimize the proposed
problem. Theoretical analysis of the optimization algorithm demonstrates the
convergence of the primal solution sequence and bounds for both the loss
function and the debiasing constraint. Uncertainty and belief-based novelty
estimation mechanisms are formulated to detect novel actions. Extensive
experiments on two real-world video datasets show that our proposed approach
achieves promising performance in single/multi-actor, single/multi-action
settings.

提出了一种基于 MUlti-Label Evidential 学习的方法，使用 Beta Evidential 神经网络评估多种不确定性，添加证据去偏执约束来优化模型，并结合不确定性和置信度机制来检测新颖行为，实验表明这一方法在单 / 多演员，单 / 多动作的情况下实现了良好的性能。

基于多标签证据学习的开放集行为识别

Open Set Action Recognition via Multi-Label Evidential Learning

In recent years, the performance of action recognition has been significantly
improved with the help of deep neural networks. Most of the existing action
recognition works hold the \textit{closed-set} assumption that all action
categories are known beforehand while deep networks can be well trained for
these categories. However, action recognition in the real world is essentially
an \textit{open-set} problem, namely, it is impossible to know all action
categories beforehand and consequently infeasible to prepare sufficient
training samples for those emerging categories. In this case, applying
closed-set recognition methods will definitely lead to unseen-category errors.
To address this challenge, we propose the Open Deep Network (ODN) for the
open-set action recognition task. Technologically, ODN detects new categories
by applying a multi-class triplet thresholding method, and then dynamically
reconstructs the classification layer and "opens" the deep network by adding
predictors for new categories continually. In order to transfer the learned
knowledge to the new category, two novel methods, Emphasis Initialization and
Allometry Training, are adopted to initialize and incrementally train the new
predictor so that only few samples are needed to fine-tune the model. Extensive
experiments show that ODN can effectively detect and recognize new categories
with little human intervention, thus applicable to the open-set action
recognition tasks in the real world. Moreover, ODN can even achieve comparable
performance to some closed-set methods.

本篇文章提出了一种新型 Open Deep Network (ODN) 架构，用于处理 Open-set action recognition 中的新类别检测和识别问题，并且可以更好地适应实际应用场景，在实验中达到了很优秀的表现。