Text-to-image diffusion models have impactful applications in art, design, and entertainment, yet these technologies also pose significant risks by enabling the creation and dissemination of misinformation. Although recent advancements have produced AI-generated image detectors that claim robustness against various augmentations, their true effectiveness remains uncertain. Do these detectors reliably identify images with different levels of augmentation? Are they biased toward specific scenes or data distributions? To investigate, we introduce SEMI-TRUTHS, featuring 27,600 real images, 223,400 masks, and 1,472,700 AI-augmented images that feature targeted and localized perturbations produced using diverse augmentation techniques, diffusion models, and data distributions. Each augmented image is accompanied by metadata for standardized and targeted evaluation of detector robustness. Our findings suggest that state-of-the-art detectors exhibit varying sensitivities to the types and degrees of perturbations, data distributions, and augmentation methods used, offering new insights into their performance and limitations. The code for the augmentation and evaluation pipeline is available at https://github.com/J-Kruk/SemiTruths.

本研究解决了AI增强图像检测器在识别不同增强水平的图像时的有效性问题。通过推出SEMI-TRUTHS数据集，包含27,600张真实图像和大量增强图像，提供了一种针对检测器鲁棒性标准化评估的新方法。研究结果表明，现有检测器对各种扰动和增强方法的敏感性参差不齐，为进一步优化图像检测技术提供了新的见解。

半真相：用于评估AI生成图像检测器鲁棒性的中大型AI增强图像数据集