Face manipulation techniques develop rapidly and arouse widespread public
concerns. Despite that vanilla convolutional neural networks achieve acceptable
performance, they suffer from the overfitting issue. To relieve this issue,
there is a trend to introduce some erasing-based augmentations. We find that
these methods indeed attempt to implicitly induce more consistent
representations for different augmentations via assigning the same label for
different augmented images. However, due to the lack of explicit
regularization, the consistency between different representations is less
satisfactory. Therefore, we constrain the consistency of different
representations explicitly and propose a simple yet effective framework,
COnsistent REpresentation Learning (CORE). Specifically, we first capture the
different representations with different augmentations, then regularize the
cosine distance of the representations to enhance the consistency. Extensive
experiments (in-dataset and cross-dataset) demonstrate that CORE performs
favorably against state-of-the-art face forgery detection methods.

针对人脸篡改技术的面部伪造检测方法，提出了一种用于增强不同表现形式下的一致性的学习框架（CORE），该框架基于卷积神经网络，通过正则化追求更一致的表示以进行更准确的伪造检测。

CORE: 人脸篡改检测中的一致性表示学习

CORE: Consistent Representation Learning for Face Forgery Detection

Recent advances in contrastive representation learning over paired image-text
data have led to models such as CLIP that achieve state-of-the-art performance
for zero-shot classification and distributional robustness. Such models
typically require joint reasoning in the image and text representation spaces
for downstream inference tasks. Contrary to prior beliefs, we demonstrate that
the image and text representations learned via a standard contrastive objective
are not interchangeable and can lead to inconsistent downstream predictions. To
mitigate this issue, we formalize consistency and propose CyCLIP, a framework
for contrastive representation learning that explicitly optimizes for the
learned representations to be geometrically consistent in the image and text
space. In particular, we show that consistent representations can be learned by
explicitly symmetrizing (a) the similarity between the two mismatched
image-text pairs (cross-modal consistency); and (b) the similarity between the
image-image pair and the text-text pair (in-modal consistency). Empirically, we
show that the improved consistency in CyCLIP translates to significant gains
over CLIP, with gains ranging from 10%-24% for zero-shot classification
accuracy on standard benchmarks (CIFAR-10, CIFAR-100, ImageNet1K) and 10%-27%
for robustness to various natural distribution shifts. The code is available at
this https URL

介绍了一种改进的对比表示学习框架 CyCLIP，通过显式对图像和文本空间的几何一致性进行优化，提高了模型的一致性和表现在零样本分类和分布不变性方面。在标准基准测试中，与 CLIP 相比，零样本分类准确率和对自然分布变化的鲁棒性分别提高了 10%-24% 和 10%-27%。