The deepfake threats to society and cybersecurity have provoked significant
public apprehension, driving intensified efforts within the realm of deepfake
video detection. Current video-level methods are mostly based on {3D CNNs}
resulting in high computational demands, although have achieved good
performance. This paper introduces an elegantly simple yet effective strategy
named Thumbnail Layout (TALL), which transforms a video clip into a pre-defined
layout to realize the preservation of spatial and temporal dependencies. This
transformation process involves sequentially masking frames at the same
positions within each frame. These frames are then resized into sub-frames and
reorganized into the predetermined layout, forming thumbnails. TALL is
model-agnostic and has remarkable simplicity, necessitating only minimal code
modifications. Furthermore, we introduce a graph reasoning block (GRB) and
semantic consistency (SC) loss to strengthen TALL, culminating in TALL++. GRB
enhances interactions between different semantic regions to capture
semantic-level inconsistency clues. The semantic consistency loss imposes
consistency constraints on semantic features to improve model generalization
ability. Extensive experiments on intra-dataset, cross-dataset,
diffusion-generated image detection, and deepfake generation method recognition
show that TALL++ achieves results surpassing or comparable to the
state-of-the-art methods, demonstrating the effectiveness of our approaches for
various deepfake detection problems. The code is available at
this https URL

这篇研究论文介绍了一个简洁而有效的策略 TALL，通过将视频剪辑转化为预定义的布局以实现空间和时间依赖的保留。通过引入图推理块（GRB）和语义一致性（SC）损失来加强 TALL，并取得了优于或与最先进方法相媲美的结果，在各种深度伪造检测问题上展示了我们方法的有效性。

基于缩略图布局和图推理的通用深伪视频检测

Towards Generalizable Deepfake Video Detection with Thumbnail Layout and  Graph Reasoning

Occlusion is a common issue in 3D reconstruction from RGB-D videos, often
blocking the complete reconstruction of objects and presenting an ongoing
problem. In this paper, we propose a novel framework, empowered by a 2D
diffusion-based in-painting model, to reconstruct complete surfaces for the
hidden parts of objects. Specifically, we utilize a pre-trained diffusion model
to fill in the hidden areas of 2D images. Then we use these in-painted images
to optimize a neural implicit surface representation for each instance for 3D
reconstruction. Since creating the in-painting masks needed for this process is
tricky, we adopt a human-in-the-loop strategy that involves very little human
engagement to generate high-quality masks. Moreover, some parts of objects can
be totally hidden because the videos are usually shot from limited
perspectives. To ensure recovering these invisible areas, we develop a cascaded
network architecture for predicting signed distance field, making use of
different frequency bands of positional encoding and maintaining overall
smoothness. Besides the commonly used rendering loss, Eikonal loss, and
silhouette loss, we adopt a CLIP-based semantic consistency loss to guide the
surface from unseen camera angles. Experiments on ScanNet scenes show that our
proposed framework achieves state-of-the-art accuracy and completeness in
object-level reconstruction from scene-level RGB-D videos.

本文提出了一个新颖的框架，利用基于扩散的 2D 修复模型，通过填补物体隐藏部分的 2D 图像来重建完整的表面，以及通过神经隐式表面表示优化每个实例的 3D 重建。实验结果表明，我们的方法在从 RGB-D 视频中进行物体级重建时达到了最先进的准确性和完整性。

O^2-Recon：利用预训练的 2D 扩散模型完成场景中被遮挡物体的 3D 重建

O^2-Recon: Completing 3D Reconstruction of Occluded Objects in the Scene  with a Pre-trained 2D Diffusion Model

We present a framework for translating unlabeled images from one domain into
analog images in another domain. We employ a progressively growing
skip-connected encoder-generator structure and train it with a GAN loss for
realistic output, a cycle consistency loss for maintaining same-domain
translation identity, and a semantic consistency loss that encourages the
network to keep the input semantic features in the output. We apply our
framework on the task of translating face images, and show that it is capable
of learning semantic mappings for face images with no supervised one-to-one
image mapping.

本论文提出了一种框架，用于将一个领域的未标记图像转换为另一个领域的类似图像，并使用逐渐增长的跳跃连接编码器 - 生成器结构进行训练，其中应用了 GAN Loss、循环一致性 Loss 和语义一致性 Loss 来保持相同领域的翻译身份，并鼓励网络保留输入的语义特征。作者在人脸图像翻译任务上应用了该框架，并展示了它可以在没有监督一对一图像映射情况下学习人脸图像的语义映射。