Advancements in deep image synthesis techniques, such as generative
adversarial networks (GANs) and diffusion models (DMs), have ushered in an era
of generating highly realistic images. While this technological progress has
captured significant interest, it has also raised concerns about the potential
difficulty in distinguishing real images from their synthetic counterparts.
This paper takes inspiration from the potent convergence capabilities between
vision and language, coupled with the zero-shot nature of vision-language
models (VLMs). We introduce an innovative method called Bi-LORA that leverages
VLMs, combined with low-rank adaptation (LORA) tuning techniques, to enhance
the precision of synthetic image detection for unseen model-generated images.
The pivotal conceptual shift in our methodology revolves around reframing
binary classification as an image captioning task, leveraging the distinctive
capabilities of cutting-edge VLM, notably bootstrapping language image
pre-training (BLIP2). Rigorous and comprehensive experiments are conducted to
validate the effectiveness of our proposed approach, particularly in detecting
unseen diffusion-generated images from unknown diffusion-based generative
models during training, showcasing robustness to noise, and demonstrating
generalization capabilities to GANs. The obtained results showcase an
impressive average accuracy of 93.41% in synthetic image detection on unseen
generation models. The code and models associated with this research can be
publicly accessed at this https URL

该研究介绍了一种创新方法 Bi-LORA，利用视觉语言模型与低秩适应技术相结合，将二分类问题重新构建为图像字幕任务，以提高对未知模型生成图像的合成图像检测精度。实验证明该方法在检测合成图像上具有 93.41% 的平均准确率。

Bi-LORA: 一种综合图像检测的视觉语言方法

Bi-LORA: A Vision-Language Approach for Synthetic Image Detection

We present a new method for synthesizing high-resolution photo-realistic
images from semantic label maps using conditional generative adversarial
networks (conditional GANs). Conditional GANs have enabled a variety of
applications, but the results are often limited to low-resolution and still far
from realistic. In this work, we generate 2048x1024 visually appealing results
with a novel adversarial loss, as well as new multi-scale generator and
discriminator architectures. Furthermore, we extend our framework to
interactive visual manipulation with two additional features. First, we
incorporate object instance segmentation information, which enables object
manipulations such as removing/adding objects and changing the object category.
Second, we propose a method to generate diverse results given the same input,
allowing users to edit the object appearance interactively. Human opinion
studies demonstrate that our method significantly outperforms existing methods,
advancing both the quality and the resolution of deep image synthesis and
editing.

使用条件生成对抗网络（conditional GANs）从语义标签图中合成高分辨率照片般真实的图像，并通过使用新的对抗性损失、多尺度生成器和鉴别器体系结构，生成 2048x1024 的视觉吸引力结果。此外，我们将我们的框架扩展到与两个额外功能的交互式视觉操作，它们是：将对象实例分割信息整合进去以进行对象操作（如删除 / 添加对象和更改对象类别），以及提出一种方法，可以在给定相同输入的情况下，生成不同的结果，允许用户交互式地编辑对象外观。人的意见研究表明，我们的方法在深层图像合成和编辑方面的质量和分辨率方面都显着优于现有方法。

基于有条件的 GAN 的高分辨率图像合成和语义操作

High-Resolution Image Synthesis and Semantic Manipulation with  Conditional GANs

In this paper, we investigate deep image synthesis guided by sketch, color,
and texture. Previous image synthesis methods can be controlled by sketch and
color strokes but we are the first to examine texture control. We allow a user
to place a texture patch on a sketch at arbitrary locations and scales to
control the desired output texture. Our generative network learns to synthesize
objects consistent with these texture suggestions. To achieve this, we develop
a local texture loss in addition to adversarial and content loss to train the
generative network. We conduct experiments using sketches generated from real
images and textures sampled from a separate texture database and results show
that our proposed algorithm is able to generate plausible images that are
faithful to user controls. Ablation studies show that our proposed pipeline can
generate more realistic images than adapting existing methods directly.

本文介绍了一种通过草图、颜色和纹理生成深层次图像的方法。通过引入纹理控制来实现用户对输出纹理的控制，我们的生成网络可以学习到如何根据用户的纹理想法生成与之一致的对象，并且在实验中得到了验证。