Generalizing to novel classes unseen during training is a key challenge of
few-shot classification. Recent metric-based methods try to address this by
local representations. However, they are unable to take full advantage of them
due to (i) improper supervision for pretraining the feature extractor, and (ii)
lack of adaptability in the metric for handling various possible compositions
of local feature sets. In this work, we unleash the power of local
representations in improving novel-class generalization. For the feature
extractor, we design a novel pretraining paradigm that learns randomly cropped
patches by soft labels. It utilizes the class-level diversity of patches while
diminishing the impact of their semantic misalignments to hard labels. To align
network output with soft labels, we also propose a UniCon KL-Divergence that
emphasizes the equal contribution of each base class in describing "non-base"
patches. For the metric, we formulate measuring local feature sets as an
entropy-regularized optimal transport problem to introduce the ability to
handle sets consisting of homogeneous elements. Furthermore, we design a
Modulate Module to endow the metric with the necessary adaptability. Our method
achieves new state-of-the-art performance on three popular benchmarks.
Moreover, it exceeds state-of-the-art transductive and cross-modal methods in
the fine-grained scenario.

利用局部表示在改进新类别通用性方面发挥了作用，我们设计了一个新的预训练范式，通过软标签学习随机裁剪的补丁，并且我们提出了一种度量方法来处理由同质元素组成的集合，我们的方法在三个常用基准测试中取得了最新的最佳性能。

发掘局部表征的威力用于少样本分类

Unleash the Power of Local Representations for Few-Shot Classification

Deepfake technology has given rise to a spectrum of novel and compelling
applications. Unfortunately, the widespread proliferation of high-fidelity fake
videos has led to pervasive confusion and deception, shattering our faith that
seeing is believing. One aspect that has been overlooked so far is that current
deepfake detection approaches may easily fall into the trap of overfitting,
focusing only on forgery clues within one or a few local regions. Moreover,
existing works heavily rely on neural networks to extract forgery features,
lacking theoretical constraints guaranteeing that sufficient forgery clues are
extracted and superfluous features are eliminated. These deficiencies culminate
in unsatisfactory accuracy and limited generalizability in real-life scenarios.
In this paper, we try to tackle these challenges through three designs: (1)
We present a novel framework to capture broader forgery clues by extracting
multiple non-overlapping local representations and fusing them into a global
semantic-rich feature. (2) Based on the information bottleneck theory, we
derive Local Information Loss to guarantee the orthogonality of local
representations while preserving comprehensive task-relevant information. (3)
Further, to fuse the local representations and remove task-irrelevant
information, we arrive at a Global Information Loss through the theoretical
analysis of mutual information. Empirically, our method achieves
state-of-the-art performance on five benchmark datasets.Our code is available
at https://github.com/QingyuLiu/Exposing-the-Deception, hoping to inspire
researchers.

通过提取多个非重叠的局部表征并将它们融合成全局的语义丰富特征，凭借信息瓶颈理论派生局部信息损失，保证局部表征的正交性同时保留全面的任务相关信息，通过互信息的理论分析得到全局信息损失，最终达到融合局部表征并且去除任务无关信息的目的，实证结果在五个基准数据集上取得了最新的性能，我们的代码可以在 https://github.com/QingyuLiu/Exposing-the-Deception 访问，希望能够激发研究人员的灵感。

揭示欺诈行为：深度伪造检测中的更多伪造线索揭示

Exposing the Deception: Uncovering More Forgery Clues for Deepfake  Detection

Despite their renowned predictive power on i.i.d. data, convolutional neural
networks are known to rely more on high-frequency patterns that humans deem
superficial than on low-frequency patterns that agree better with intuitions
about what constitutes category membership. This paper proposes a method for
training robust convolutional networks by penalizing the predictive power of
the local representations learned by earlier layers. Intuitively, our networks
are forced to discard predictive signals such as color and texture that can be
gleaned from local receptive fields and to rely instead on the global
structures of the image. Across a battery of synthetic and benchmark domain
adaptation tasks, our method confers improved generalization out of the domain.
Also, to evaluate cross-domain transfer, we introduce ImageNet-Sketch, a new
dataset consisting of sketch-like images, that matches the ImageNet
classification validation set in categories and scale.

本文提出了一种训练且具有鲁棒性的卷积神经网络的方法，通过惩罚较早层所学习的局部表示的预测能力，迫使网络丢弃色彩和纹理等可从局部感受野中获取的预测信号，并依靠图像的全局结构。在一系列综合性和基准领域适应任务中，我们的方法使泛化能力得到了改善。我们还引入了一个新的数据集 ImageNet-Sketch，以评估跨域传输。