The underground exploitation of large language models (LLMs) for malicious
services (i.e., Malla) is witnessing an uptick, amplifying the cyber threat
landscape and posing questions about the trustworthiness of LLM technologies.
However, there has been little effort to understand this new cybercrime, in
terms of its magnitude, impact, and techniques. In this paper, we conduct the
first systematic study on 212 real-world Mallas, uncovering their proliferation
in underground marketplaces and exposing their operational modalities. Our
study discloses the Malla ecosystem, revealing its significant growth and
impact on today's public LLM services. Through examining 212 Mallas, we
uncovered eight backend LLMs used by Mallas, along with 182 prompts that
circumvent the protective measures of public LLM APIs. We further demystify the
tactics employed by Mallas, including the abuse of uncensored LLMs and the
exploitation of public LLM APIs through jailbreak prompts. Our findings enable
a better understanding of the real-world exploitation of LLMs by
cybercriminals, offering insights into strategies to counteract this
cybercrime.

通过对 212 个真实世界 Malla 的研究，我们揭示了地下市场中 Malla 的快速增长和对当今公共 LLM 服务的影响，发现了 Malla 生态系统并暴露了它们的运作方式，包括 8 个后端 LLM 和 182 个绕过公共 LLM API 保护措施的提示，同时解密了 Malla 所采用的战术，如滥用未经审查的 LLM 和通过越狱提示利用公共 LLM API。我们的研究有助于更好地了解网络犯罪分子对 LLM 的实际利用，并为应对这种网络犯罪提供策略洞察。