Molecular biologists frequently interpret gene lists derived from
high-throughput experiments and computational analysis. This is typically done
as a statistical enrichment analysis that measures the over- or
under-representation of biological function terms associated with genes or
their properties, based on curated assertions from a knowledge base (KB) such
as the Gene Ontology (GO). Interpreting gene lists can also be framed as a
textual summarization task, enabling the use of Large Language Models (LLMs),
potentially utilizing scientific texts directly and avoiding reliance on a KB.
We developed SPINDOCTOR (Structured Prompt Interpolation of Natural Language
Descriptions of Controlled Terms for Ontology Reporting), a method that uses
GPT models to perform gene set function summarization as a complement to
standard enrichment analysis. This method can use different sources of gene
functional information: (1) structured text derived from curated ontological KB
annotations, (2) ontology-free narrative gene summaries, or (3) direct model
retrieval.
We demonstrate that these methods are able to generate plausible and
biologically valid summary GO term lists for gene sets. However, GPT-based
approaches are unable to deliver reliable scores or p-values and often return
terms that are not statistically significant. Crucially, these methods were
rarely able to recapitulate the most precise and informative term from standard
enrichment, likely due to an inability to generalize and reason using an
ontology. Results are highly nondeterministic, with minor variations in prompt
resulting in radically different term lists. Our results show that at this
point, LLM-based methods are unsuitable as a replacement for standard term
enrichment analysis and that manual curation of ontological assertions remains
necessary.

本研究提出了一种基于 GPT 模型的新的基因集功能摘要方法，该方法使用结构化文本、自由叙述基因概要或直接模型检索等来源来生成 GO 术语列表，但并不能依靠本方法来代替标准术语富集分析，手动筛选方法仍然必要。