Digital image forensics plays a crucial role in image authentication and
manipulation localization. Despite the progress powered by deep neural
networks, existing forgery localization methodologies exhibit limitations when
deployed to unseen datasets and perturbed images (i.e., lack of generalization
and robustness to real-world applications). To circumvent these problems and
aid image integrity, this paper presents a generalized and robust manipulation
localization model through the analysis of pixel inconsistency artifacts. The
rationale is grounded on the observation that most image signal processors
(ISP) involve the demosaicing process, which introduces pixel correlations in
pristine images. Moreover, manipulating operations, including splicing,
copy-move, and inpainting, directly affect such pixel regularity. We,
therefore, first split the input image into several blocks and design masked
self-attention mechanisms to model the global pixel dependency in input images.
Simultaneously, we optimize another local pixel dependency stream to mine local
manipulation clues within input forgery images. In addition, we design novel
Learning-to-Weight Modules (LWM) to combine features from the two streams,
thereby enhancing the final forgery localization performance. To improve the
training process, we propose a novel Pixel-Inconsistency Data Augmentation
(PIDA) strategy, driving the model to focus on capturing inherent pixel-level
artifacts instead of mining semantic forgery traces. This work establishes a
comprehensive benchmark integrating 15 representative detection models across
12 datasets. Extensive experiments show that our method successfully extracts
inherent pixel-inconsistency forgery fingerprints and achieve state-of-the-art
generalization and robustness performances in image manipulation localization.

该研究提出了一个通过像素不一致性痕迹进行广义且稳健的图像篡改定位的方法，通过分析像素不一致性痕迹模型化图像的全局像素依赖和本地篡改线索，并通过学习权重模块增强最终的篡改定位性能。在图像信号处理器插值过程中引入的像素相关性以及篡改操作对像素的影响是该方法的基础，并通过新颖的数据增强策略驱使模型专注于捕捉固有的像素级痕迹。实验结果表明，该方法在图像篡改定位领域具有最先进的泛化和稳健性能。

像素不一致性建模用于图像篡改定位

Pixel-Inconsistency Modeling for Image Manipulation Localization

Deep Image Manipulation Localization (IML) models suffer from training data
insufficiency and thus heavily rely on pre-training. We argue that contrastive
learning is more suitable to tackle the data insufficiency problem for IML.
Crafting mutually exclusive positives and negatives is the prerequisite for
contrastive learning. However, when adopting contrastive learning in IML, we
encounter three categories of image patches: tampered, authentic, and contour
patches. Tampered and authentic patches are naturally mutually exclusive, but
contour patches containing both tampered and authentic pixels are non-mutually
exclusive to them. Simply abnegating these contour patches results in a drastic
performance loss since contour patches are decisive to the learning outcomes.
Hence, we propose the Non-mutually exclusive Contrastive Learning (NCL)
framework to rescue conventional contrastive learning from the above dilemma.
In NCL, to cope with the non-mutually exclusivity, we first establish a pivot
structure with dual branches to constantly switch the role of contour patches
between positives and negatives while training. Then, we devise a
pivot-consistent loss to avoid spatial corruption caused by the role-switching
process. In this manner, NCL both inherits the self-supervised merits to
address the data insufficiency and retains a high manipulation localization
accuracy. Extensive experiments verify that our NCL achieves state-of-the-art
performance on all five benchmarks without any pre-training and is more robust
on unseen real-life samples. The code is available at:
this https URL

将非互斥对比学习（NCL）引入深度图像处理定位（IML）模型，以解决训练数据不足的问题，并提高处理位置准确度。NCL 通过建立一个包含双分支的中心结构，并设计一种保持空间上连续性的损失函数，有效地应对不互斥图像区域的问题，实现了自我监督的优势并在五个基准测试中表现出卓越性能。

无预训练图像操纵定位的非互斥对比学习

Pre-training-free Image Manipulation Localization through Non-Mutually  Exclusive Contrastive Learning

With advanced image journaling tools, one can easily alter the semantic
meaning of an image by exploiting certain manipulation techniques such as
copy-clone, object splicing, and removal, which mislead the viewers. In
contrast, the identification of these manipulations becomes a very challenging
task as manipulated regions are not visually apparent. This paper proposes a
high-confidence manipulation localization architecture which utilizes
resampling features, Long-Short Term Memory (LSTM) cells, and encoder-decoder
network to segment out manipulated regions from non-manipulated ones.
Resampling features are used to capture artifacts like JPEG quality loss,
upsampling, downsampling, rotation, and shearing. The proposed network exploits
larger receptive fields (spatial maps) and frequency domain correlation to
analyze the discriminative characteristics between manipulated and
non-manipulated regions by incorporating encoder and LSTM network. Finally,
decoder network learns the mapping from low-resolution feature maps to
pixel-wise predictions for image tamper localization. With predicted mask
provided by final layer (softmax) of the proposed architecture, end-to-end
training is performed to learn the network parameters through back-propagation
using ground-truth masks. Furthermore, a large image splicing dataset is
introduced to guide the training process. The proposed method is capable of
localizing image manipulations at pixel level with high precision, which is
demonstrated through rigorous experimentation on three diverse datasets.

本文提出了一种基于图像局部特征和深度学习的高置信度图像篡改检测和定位算法，通过利用重采样特征、长短期记忆单元和编码器 - 解码器网络实现了对篡改区域和非篡改区域的有效分割定位，提出的方法能够在像素级非常准确地定位图像篡改。