Open-set Recognition (OSR) aims to identify test samples whose classes are
not seen during the training process. Recently, Unified Open-set Recognition
(UOSR) has been proposed to reject not only unknown samples but also known but
wrongly classified samples, which tends to be more practical in real-world
applications. The UOSR draws little attention since it is proposed, but we find
sometimes it is even more practical than OSR in the real world applications, as
evaluation results of known but wrongly classified samples are also wrong like
unknown samples. In this paper, we deeply analyze the UOSR task under different
training and evaluation settings to shed light on this promising research
direction. For this purpose, we first evaluate the UOSR performance of several
OSR methods and show a significant finding that the UOSR performance
consistently surpasses the OSR performance by a large margin for the same
method. We show that the reason lies in the known but wrongly classified
samples, as their uncertainty distribution is extremely close to unknown
samples rather than known and correctly classified samples. Second, we analyze
how the two training settings of OSR (i.e., pre-training and outlier exposure)
influence the UOSR. We find although they are both beneficial for
distinguishing known and correctly classified samples from unknown samples,
pre-training is also helpful for identifying known but wrongly classified
samples while outlier exposure is not. In addition to different training
settings, we also formulate a new evaluation setting for UOSR which is called
few-shot UOSR, where only one or five samples per unknown class are available
during evaluation to help identify unknown samples. We propose FS-KNNS for the
few-shot UOSR to achieve state-of-the-art performance under all settings.

本文深入分析了 UOSR 任务在不同训练和评估设置下的性能，并评估了几种 OSR 方法的 UOSR 性能。我们发现 UOSR 性能明显优于相同方法的 OSR 性能，原因在于已知但错误分类的样本。我们还探讨了 OSR 的两种训练设置如何影响 UOSR，提出了新的评估设置 few-shot UOSR，并提出了 FS-KNNS 来实现在所有设置下的最先进性能。