Empathetic dialogue is an indispensable part of building harmonious social
relationships and contributes to the development of a helpful AI. Previous
approaches are mainly based on fine small-scale language models. With the
advent of ChatGPT, the application effect of large language models (LLMs) in
this field has attracted great attention. This work empirically investigates
the performance of LLMs in generating empathetic responses and proposes three
improvement methods of semantically similar in-context learning, two-stage
interactive generation, and combination with the knowledge base. Extensive
experiments show that LLMs can significantly benefit from our proposed methods
and is able to achieve state-of-the-art performance in both automatic and human
evaluations. Additionally, we explore the possibility of GPT-4 simulating human
evaluators.

大型语言模型在培养有同理心的对话、构建和谐社会关系以及发展有帮助的人工智能方面具有至关重要的作用。本研究通过实证调查了大型语言模型在生成有同理心的回应方面的性能，并提出了三种改进方法：语义上下文学习、两阶段交互生成和与知识库的结合。广泛的实验证明，我们的方法能够显著改善大型语言模型的性能，在自动评估和人工评估方面达到最先进的水平。此外，我们还探索了 GPT-4 模型模拟人工评估者的可能性。