In many scientific fields, large language models (LLMs) have revolutionized
the way with which text and other modalities of data (e.g., molecules and
proteins) are dealt, achieving superior performance in various applications and
augmenting the scientific discovery process. Nevertheless, previous surveys on
scientific LLMs often concentrate on one to two fields or a single modality. In
this paper, we aim to provide a more holistic view of the research landscape by
unveiling cross-field and cross-modal connections between scientific LLMs
regarding their architectures and pre-training techniques. To this end, we
comprehensively survey over 250 scientific LLMs, discuss their commonalities
and differences, as well as summarize pre-training datasets and evaluation
tasks for each field and modality. Moreover, we investigate how LLMs have been
deployed to benefit scientific discovery. Resources related to this survey are
available at this https URL

该研究对 250 个科学领域的大型语言模型进行了全面调研，揭示了它们在架构和预训练技术上的交叉领域和跨模态连接，并总结了每个领域和模态的预训练数据集和评估任务。此外，还研究了大型语言模型在科学发现中的应用。

科学大型语言模型及其在科学发现中的应用综述

A Comprehensive Survey of Scientific Large Language Models and Their  Applications in Scientific Discovery

Large Language Models (LLMs) have emerged as a transformative power in
enhancing natural language comprehension, representing a significant stride
toward artificial general intelligence. The application of LLMs extends beyond
conventional linguistic boundaries, encompassing specialized linguistic systems
developed within various scientific disciplines. This growing interest has led
to the advent of scientific LLMs, a novel subclass specifically engineered for
facilitating scientific discovery. As a burgeoning area in the community of AI
for Science, scientific LLMs warrant comprehensive exploration. However, a
systematic and up-to-date survey introducing them is currently lacking. In this
paper, we endeavor to methodically delineate the concept of "scientific
language", whilst providing a thorough review of the latest advancements in
scientific LLMs. Given the expansive realm of scientific disciplines, our
analysis adopts a focused lens, concentrating on the biological and chemical
domains. This includes an in-depth examination of LLMs for textual knowledge,
small molecules, macromolecular proteins, genomic sequences, and their
combinations, analyzing them in terms of model architectures, capabilities,
datasets, and evaluation. Finally, we critically examine the prevailing
challenges and point out promising research directions along with the advances
of LLMs. By offering a comprehensive overview of technical developments in this
field, this survey aspires to be an invaluable resource for researchers
navigating the intricate landscape of scientific LLMs.

大型语言模型 (LLMs) 在增强自然语言理解方面具有变革性的力量，代表着朝着人工通用智能迈出的重要一步。科学 LLMs 是一个新兴的领域，专门针对促进科学发现进行工程化设计。本文详细调查了科学 LLMs，并围绕生物化学领域进行了深入研究，包括文本知识、小型分子、大分子蛋白质、基因组序列以及它们的组合，从模型架构、能力、数据集和评估等方面进行了分析。最后，我们批判性地审查了当前的挑战，并指出了前景广阔的研究方向以及 LLMs 的进展。通过提供对该领域技术发展的全面概述，本调查力求成为研究人员在科学 LLMs 复杂领域中的宝贵资源。