Large language models (LLMs) hold great potential for many natural language
applications but risk generating incorrect or toxic content. To probe when an
LLM generates unwanted content, the current paradigm is to recruit a
\textit{red team} of human testers to design input prompts (i.e., test cases)
that elicit undesirable responses from LLMs. However, relying solely on human
testers is expensive and time-consuming. Recent works automate red teaming by
training a separate red team LLM with reinforcement learning (RL) to generate
test cases that maximize the chance of eliciting undesirable responses from the
target LLM. However, current RL methods are only able to generate a small
number of effective test cases resulting in a low coverage of the span of
prompts that elicit undesirable responses from the target LLM. To overcome this
limitation, we draw a connection between the problem of increasing the coverage
of generated test cases and the well-studied approach of curiosity-driven
exploration that optimizes for novelty. Our method of curiosity-driven red
teaming (CRT) achieves greater coverage of test cases while mantaining or
increasing their effectiveness compared to existing methods. Our method, CRT
successfully provokes toxic responses from LLaMA2 model that has been heavily
fine-tuned using human preferences to avoid toxic outputs. Code is available at
https://github.com/Improbable-AI/curiosity_redteam

通过好奇心驱动的红队（CRT），我们提出了一种自动生成测试用例的方法，以增加生成的测试用例的覆盖范围，并成功地从经过重度优化以避免有害结果的 LLaMA2 模型中引发有害回应。

大型语言模型的好奇心驱动的红队扮演

Curiosity-driven Red-teaming for Large Language Models

Automated test case generation has proven to be useful to reduce the usually
high expenses of software testing. However, several studies have also noted the
skepticism of testers regarding the comprehension of generated test suites when
compared to manually designed ones. This fact suggests that involving testers
in the test generation process could be helpful to increase their acceptance of
automatically-produced test suites. In this paper, we propose incorporating
interactive readability assessments made by a tester into EvoSuite, a
widely-known evolutionary test generation tool. Our approach, InterEvo-TR,
interacts with the tester at different moments during the search and shows
different test cases covering the same coverage target for their subjective
evaluation. The design of such an interactive approach involves a schedule of
interaction, a method to diversify the selected targets, a plan to save and
handle the readability values, and some mechanisms to customize the level of
engagement in the revision, among other aspects. To analyze the potential and
practicability of our proposal, we conduct a controlled experiment in which 39
participants, including academics, professional developers, and student
collaborators, interact with InterEvo-TR. Our results show that the strategy to
select and present intermediate results is effective for the purpose of
readability assessment. Furthermore, the participants' actions and responses to
a questionnaire allowed us to analyze the aspects influencing test code
readability and the benefits and limitations of an interactive approach in the
context of test case generation, paving the way for future developments based
on interactivity.

自动生成测试用例的自动化方法在软件测试中降低了成本，但测试人员对于生成的测试套件的理解相对于手动设计的测试套件持怀疑态度。为了增强测试人员接受自动生成的测试套件，本研究提出了一种将测试人员的互动可读性评估纳入 EvoSuite（一个广为人知的进化式测试生成工具）的方法。通过与 39 名参与者进行的控制实验，研究结果表明选择和呈现中间结果的策略对于可读性评估是有效的。此外，参与者的行为和对问卷的回答使我们能够分析影响测试代码可读性的因素以及互动方法在测试用例生成环境中的益处和限制，为基于互动性的未来研究提供了奠基。