Large Language Models (LLMs) are often asked to explain their outputs to enhance accuracy and transparency. However, evidence suggests that these explanations can misrepresent the models' true reasoning processes. One effective way to identify inaccuracies or omissions in these explanations is through consistency checking, which typically involves asking follow-up questions. This paper introduces, cross-examiner, a new method for generating follow-up questions based on a model's explanation of an initial question. Our method combines symbolic information extraction with language model-driven question generation, resulting in better follow-up questions than those produced by LLMs alone. Additionally, this approach is more flexible than other methods and can generate a wider variety of follow-up questions.

本文解决了大型语言模型生成的解释可能误导用户真实推理过程的问题。提出了一种新的方法交叉审查者，通过结合符号信息提取与语言模型驱动的问题生成，生成更优质的后续问题。研究发现，该方法在灵活性和后续问题的多样性上优于现有方法，具有重要的潜在影响。

交叉审查者：评估大型语言模型生成解释的一致性