Recently, significant public efforts have been directed towards developing low-cost models with capabilities akin to ChatGPT, thereby fostering the growth of open-source conversational models. However, there remains a scarcity of comprehensive and in-depth evaluations of these models' performance. In this study, we examine the influence of training data factors, including quantity, quality, and linguistic distribution, on model performance. Our analysis is grounded in several publicly accessible, high-quality instruction datasets, as well as our own Chinese multi-turn conversations. We assess various models using a evaluation set of 1,000 samples, encompassing nine real-world scenarios. Our goal is to supplement manual evaluations with quantitative analyses, offering valuable insights for the continued advancement of open-source chat models. Furthermore, to enhance the performance and training and inference efficiency of models in the Chinese domain, we extend the vocabulary of LLaMA - the model with the closest open-source performance to proprietary language models like GPT-3 - and conduct secondary pre-training on 3.4B Chinese words. We make our model, data, as well as code publicly available.

本研究旨在使用公开数据集结合自身汉语多轮对话中的数据进行分析，选取各种评估指标来评价各类开源聊天机器人的性能表现，并对 LLaMA 进行词汇扩展及 34 亿汉语单词的二次预训练，以期提升聊天机器人在中文领域的表现与效率，最后将模型、数据、代码进行公开发布。

探究训练数据和评估对中文指示性语言模型的影响