Novel Categories Discovery (NCD) tackles the open-world problem of
classifying known and clustering novel categories based on the class semantics
using partial class space annotated data. Unlike traditional pseudo-label and
retraining, we investigate NCD from the novel data probability matrix
perspective. We leverage the connection between NCD novel data sampling with
provided novel class Multinoulli (categorical) distribution and hypothesize to
implicitly achieve semantic-based novel data clustering by learning their class
distribution. We propose novel constraints on first-order (mean) and
second-order (covariance) statistics of probability matrix features while
applying instance-wise information constraints. In particular, we align the
neuron distribution (activation patterns) under a large batch of Monte-Carlo
novel data sampling by matching their empirical features mean and covariance
with the provided Multinoulli-distribution. Simultaneously, we minimize entropy
and enforce prediction consistency for each instance. Our simple approach
successfully realizes semantic-based novel data clustering provided the
semantic similarity between label-unlabeled classes. We demonstrate the
discriminative capacity of our approaches in image and video modalities.
Moreover, we perform extensive ablation studies regarding data, networks, and
our framework components to provide better insights. Our approach maintains
~94%, ~93%, and ~85%, classification accuracy in labeled data while achieving
~90%, ~84%, and ~72% clustering accuracy for novel categories for Cifar10,
UCF101, and MPSC-ARL datasets that matches state-of-the-art approaches without
any external clustering.

基于部分类别空间注释数据，对已知类别进行分类和聚类及发现新类别 (Novel Categories Discovery，NCD) 解决了开放世界问题。通过硬约束和软约束相结合的方法，我们成功地实现了基于语义的新类别数据聚类。