As new research on Large Language Models (LLMs) continues, it is difficult to
keep up with new research and models. To help researchers synthesize the new
research many have written survey papers, but even those have become numerous.
In this paper, we develop a method to automatically assign survey papers to a
taxonomy. We collect the metadata of 144 LLM survey papers and explore three
paradigms to classify papers within the taxonomy. Our work indicates that
leveraging graph structure information on co-category graphs can significantly
outperform the language models in two paradigms; pre-trained language models'
fine-tuning and zero-shot/few-shot classifications using LLMs. We find that our
model surpasses an average human recognition level and that fine-tuning LLMs
using weak labels generated by a smaller model, such as the GCN in this study,
can be more effective than using ground-truth labels, revealing the potential
of weak-to-strong generalization in the taxonomy classification task.

我们开发了一种方法来自动将调查论文分配给一个分类体系，在 144 篇大型语言模型调查论文的元数据的基础上，我们探索了三种范式来分类分类体系中的论文。我们的研究表明，在两种范式中，利用共同类别图的图结构信息可以明显优于语言模型。我们发现我们的模型超过了人类的平均识别水平，而使用由较小模型生成的弱标签进行大型语言模型的微调，如本研究中的 GCN，比使用地面真实标签更有效，揭示了在分类任务中从弱到强的泛化潜力。

通过图表示学习，理解关于大型语言模型的综述论文分类

Understanding Survey Paper Taxonomy about Large Language Models via  Graph Representation Learning

We present a novel platform for evaluating the capability of Large Language
Models (LLMs) to autonomously compose and critique survey papers spanning a
vast array of disciplines including sciences, humanities, education, and law.
Within this framework, AI systems undertake a simulated peer-review mechanism
akin to traditional scholarly journals, with human organizers serving in an
editorial oversight capacity. Within this framework, we organized a competition
for the AutoML conference 2023. Entrants are tasked with presenting stand-alone
models adept at authoring articles from designated prompts and subsequently
appraising them. Assessment criteria include clarity, reference
appropriateness, accountability, and the substantive value of the content. This
paper presents the design of the competition, including the implementation
baseline submissions and methods of evaluation.

我们提出了一种新颖的平台，用于评估大型语言模型（LLMs）自主撰写和评论跨科学、人文、教育和法律等各个学科的调研论文的能力。这个框架中，人工智能系统通过模拟同行评审机制进行操作，类似于传统学术期刊，人类组织者担任编辑监督职责。在这个框架内，我们为 2023 年 AutoML 会议组织了一次竞赛。参赛者的任务是根据指定提示撰写独立的文章，并对其进行评估。评估标准包括清晰度、参考文献适当性、责任性和内容的实质价值。本文介绍了竞赛的设计，包括实施基线提交和评估方法。