As the popularity of large language models (LLMs) soars across various
applications, ensuring their alignment with human values has become a paramount
concern. In particular, given that LLMs have great potential to serve as
general-purpose AI assistants in daily life, their subtly unethical suggestions
become a serious and real concern. Tackling the challenge of automatically
testing and repairing unethical suggestions is thus demanding.
This paper introduces the first framework for testing and repairing unethical
suggestions made by LLMs. We first propose ETHICSSUITE, a test suite that
presents complex, contextualized, and realistic moral scenarios to test LLMs.
We then propose a novel suggest-critic-reflect (SCR) process, serving as an
automated test oracle to detect unethical suggestions. We recast deciding if
LLMs yield unethical suggestions (a hard problem; often requiring human
expertise and costly to decide) into a PCR task that can be automatically
checked for violation. Moreover, we propose a novel on-the-fly (OTF) repairing
scheme that repairs unethical suggestions made by LLMs in real-time. The OTF
scheme is applicable to LLMs in a black-box API setting with moderate cost.
With ETHICSSUITE, our study on seven popular LLMs (e.g., ChatGPT, GPT-4)
uncovers in total 109,824 unethical suggestions. We apply our OTF scheme on two
LLMs (Llama-13B and ChatGPT), which generates valid repair to a considerable
amount of unethical ones, paving the way for more ethically conscious LLMs.

本文提出了第一个用于测试和修复 LMMs 不道德建议的框架，包括测试套件 ETHICSSUITE、建议 - 评论 - 反思（SCR）流程、在线修复方案等，并使用 ETHICSSUITE 测试了 7 个流行的 LMM，发现了总计 109824 个不道德建议，并在 Llama-13B 和 ChatGPT 上应用了 OTF 方案，为更加注重伦理的 LMMs 铺平了道路。