In this article we report on an initial exploration to assess the viability
of using the general large language models (LLMs), recently made public, to
classify mathematical documents. Automated classification would be useful from
the applied perspective of improving the navigation of the literature and the
more open-ended goal of identifying relations among mathematical results. The
Mathematical Subject Classification MSC 2020, from MathSciNet and zbMATH, is
widely used and there is a significant corpus of ground truth material in the
open literature. We have evaluated the classification of preprint articles from
arXiv.org according to MSC 2020. The experiment used only the title and
abstract alone -- not the entire paper. Since this was early in the use of
chatbots and the development of their APIs, we report here on what was carried
out by hand. Of course, the automation of the process will have to follow if it
is to be generally useful. We found that in about 60% of our sample the LLM
produced a primary classification matching that already reported on arXiv. In
about half of those instances, there were additional primary classifications
that were not detected. In about 40% of our sample, the LLM suggested a
different classification than what was provided. A detailed examination of
these cases, however, showed that the LLM-suggested classifications were in
most cases better than those provided.

使用大型语言模型对数学文档进行自动分类，根据 Mathematical Subject Classification 对 arXiv.org 上的预印本文章进行评估，发现大约 60% 的样本中，语言模型的主要分类与 arXiv 上报告的分类相匹配，并且在其中一半的情况下，语言模型还提供了额外的主要分类。大约 40% 的样本中，语言模型提供了与原分类不同的分类，但经过详细检查发现，语言模型提供的分类大多数情况下更准确。