Recent advancements in generative artificial intelligence (AI) have raised
concerns about their potential to create convincing fake social media accounts,
but empirical evidence is lacking. In this paper, we present a systematic
analysis of Twitter(X) accounts using human faces generated by Generative
Adversarial Networks (GANs) for their profile pictures. We present a dataset of
1,353 such accounts and show that they are used to spread scams, spam, and
amplify coordinated messages, among other inauthentic activities. Leveraging a
feature of GAN-generated faces -- consistent eye placement -- and supplementing
it with human annotation, we devise an effective method for identifying
GAN-generated profiles in the wild. Applying this method to a random sample of
active Twitter users, we estimate a lower bound for the prevalence of profiles
using GAN-generated faces between 0.021% and 0.044% -- around 10K daily active
accounts. These findings underscore the emerging threats posed by multimodal
generative AI. We release the source code of our detection method and the data
we collect to facilitate further investigation. Additionally, we provide
practical heuristics to assist social media users in recognizing such accounts.

通过使用生成对抗网络 (GANs) 生成的具有人脸的推特用户作为样本，本研究系统分析了近期发展的生成人工智能 (AI) 在创建可信的假社交媒体账户方面的潜在问题。结果发现这些账户被用于传播欺诈、垃圾邮件以及增强协调的信息等虚假行为，并通过利用 GAN 生成的人脸中眼睛位置的特征以及人工注释的方法，对野外中的 GAN 生成的个人资料进行有效的识别。研究估计活跃推特用户中使用 GAN 生成的人脸的比例下限为 0.021% 至 0.044%，即每天约有 1 万个活跃账户。这些发现凸显了多模态生成 AI 所带来的新威胁。研究者公开了检测方法的源代码和收集到的数据以助于进一步调查，并提供实用启发法帮助社交媒体用户辨别此类账户。