Aspect sentiment quad prediction (ASQP) is a critical subtask of aspect-level
sentiment analysis. Current ASQP datasets are characterized by their small size
and low quadruple density, which hinders technical development. To expand
capacity, we construct two large Chinese ASQP datasets crawled from multiple
online platforms. The datasets hold several significant characteristics: larger
size (each with 10,000+ samples) and rich aspect categories, more words per
sentence, and higher density than existing ASQP datasets. Moreover, we are the
first to evaluate the performance of Generative Pre-trained Transformer (GPT)
series models on ASQP and exhibit potential issues. The experiments with
state-of-the-art ASQP baselines underscore the need to explore additional
techniques to address ASQP, as well as the importance of further investigation
into methods to improve the performance of GPTs.

通过构建两个大型的中文 aspect sentiment quad prediction（ASQP）数据集，我们评估了 GPT 系列模型在 ASQP 上的性能，并展示了潜在的问题，同时强调了需要探索额外技术以解决 ASQP 以及改善 GPT 性能的重要性。

中国方面情感四元预测基准的实证研究

An Empirical Study of Benchmarking Chinese Aspect Sentiment Quad  Prediction

GPT series models, such as GPT-3, CodeX, InstructGPT, ChatGPT, and so on,
have gained considerable attention due to their exceptional natural language
processing capabilities. However, despite the abundance of research on the
difference in capabilities between GPT series models and fine-tuned models,
there has been limited attention given to the evolution of GPT series models'
capabilities over time. To conduct a comprehensive analysis of the capabilities
of GPT series models, we select six representative models, comprising two GPT-3
series models (i.e., davinci and text-davinci-001) and four GPT-3.5 series
models (i.e., code-davinci-002, text-davinci-002, text-davinci-003, and
gpt-3.5-turbo). We evaluate their performance on nine natural language
understanding (NLU) tasks using 21 datasets. In particular, we compare the
performance and robustness of different models for each task under zero-shot
and few-shot scenarios. Our extensive experiments reveal that the overall
ability of GPT series models on NLU tasks does not increase gradually as the
models evolve, especially with the introduction of the RLHF training strategy.
While this strategy enhances the models' ability to generate human-like
responses, it also compromises their ability to solve some tasks. Furthermore,
our findings indicate that there is still room for improvement in areas such as
model robustness.

本研究选取了 6 个代表性的 GPT 系列模型，对它们在 21 个数据集上进行了 9 项自然语言任务的性能评估，发现尽管引入强化学习任务和人性化反馈机制提高了 GPT 系列模型生成人类语言的能力，但也有牺牲部分任务解决能力的代价，同时还有提高模型的鲁棒性的空间。