Recent studies have demonstrated that large language models (LLMs) excel in
diverse tasks through in-context learning (ICL) facilitated by task-specific
prompts and examples. However, the existing literature shows that ICL
encounters performance deterioration when exposed to adversarial inputs.
Enhanced performance has been observed when ICL is augmented with natural
language explanations (NLEs) (we refer to it as X-ICL). Thus, this work
investigates whether X-ICL can improve the robustness of LLMs on a suite of
seven adversarial and challenging natural language inference datasets.
Moreover, we introduce a new approach to X-ICL by prompting an LLM (ChatGPT in
our case) with few human-generated NLEs to produce further NLEs (we call it
ChatGPT few-shot), which we show superior to both ChatGPT zero-shot and
human-generated NLEs alone. We evaluate five popular LLMs (GPT3.5-turbo,
LLaMa2, Vicuna, Zephyr, Mistral) and show that X-ICL with ChatGPT few-shot
yields over 6% improvement over ICL. Furthermore, while prompt selection
strategies were previously shown to significantly improve ICL on
in-distribution test sets, we show that these strategies do not match the
efficacy of the X-ICL paradigm in robustness-oriented evaluations.

通过加上自然语言解释，人工智能模型在各种任务中有了显著提升，但在遇到对抗性输入时，效果下降。本文研究了如何通过加入自然语言解释来提高模型对七个对抗性和具有挑战性的自然语言推理数据集的鲁棒性，并提出了 ChatGPT 的几个人工生成的自然语言解释来产生更多自然语言解释的新方法，表现更好。通过对五种常见的大语言模型进行评估，我们发现 ChatGPT 的几个人工生成的自然语言解释的 X-ICL 方法比 ICL 方法提高了 6% 以上。此外，我们还发现之前能有效提高 ICL 性能的提示选择策略，在鲁棒性评估中并不具有与 X-ICL 范式相称的效果。