The Third Generation Partnership Project (3GPP) has successfully introduced
standards for global mobility. However, the volume and complexity of these
standards has increased over time, thus complicating access to relevant
information for vendors and service providers. Use of Generative Artificial
Intelligence (AI) and in particular Large Language Models (LLMs), may provide
faster access to relevant information. In this paper, we evaluate the
capability of state-of-art LLMs to be used as Question Answering (QA)
assistants for 3GPP document reference. Our contribution is threefold. First,
we provide a benchmark and measuring methods for evaluating performance of
LLMs. Second, we do data preprocessing and fine-tuning for one of these LLMs
and provide guidelines to increase accuracy of the responses that apply to all
LLMs. Third, we provide a model of our own, TeleRoBERTa, that performs on-par
with foundation LLMs but with an order of magnitude less number of parameters.
Results show that LLMs can be used as a credible reference tool on telecom
technical documents, and thus have potential for a number of different
applications from troubleshooting and maintenance, to network operations and
software product development.

通过评估最新的大型语言模型作为第三代合作伙伴计划（3GPP）文件参考的问答助手的能力，本文提供了性能评估的基准和测量方法，进行了数据预处理和微调，提供了适用于所有语言模型的增加响应准确性的指南，并提供了一种名为 TeleRoBERTa 的模型，其性能与基础语言模型相当，但参数数量少一个数量级。结果表明，大型语言模型可以作为电信技术文件的可靠参考工具，因此在故障排除、维护、网络运营和软件产品开发等多个领域具有潜力。