Previous studies have demonstrated that proactive interaction with user
reviews has a positive impact on the perception of app users and encourages
them to submit revised ratings. Nevertheless, developers encounter challenges
in managing a high volume of reviews, particularly in the case of popular apps
with a substantial influx of daily reviews. Consequently, there is a demand for
automated solutions aimed at streamlining the process of responding to user
reviews. To address this, we have developed a new system for generating
automatic responses by leveraging user-contributed documents with the help of
retrieval-augmented generation (RAG) and advanced Large Language Models (LLMs).
Our solution, named SCRABLE, represents an adaptive customer review response
automation that enhances itself with self-optimizing prompts and a judging
mechanism based on LLMs. Additionally, we introduce an automatic scoring
mechanism that mimics the role of a human evaluator to assess the quality of
responses generated in customer review domains. Extensive experiments and
analyses conducted on real-world datasets reveal that our method is effective
in producing high-quality responses, yielding improvement of more than 8.5%
compared to the baseline. Further validation through manual examination of the
generated responses underscores the efficacy our proposed system.

我们开发了一个名为 SCRABLE 的新系统，利用检索增强生成（RAG）和先进的大语言模型（LLM）从用户贡献的文档中生成自动回复，通过自优化提示和基于 LLM 的评判机制增强自身的自适应客户评价回复自动化，通过模仿人类评估者的自动评分机制评估生成回复的质量，在真实数据集上进行的广泛实验和分析表明，相比基准线，我们的方法在生成高质量回复方面有效，改进率超过 8.5%。通过对生成回复的手动检查进一步验证了我们提出的系统的效果。