Optimizing black-box functions is a fundamental problem in science and
engineering. To solve this problem, many approaches learn a surrogate function
that estimates the underlying objective from limited historical evaluations.
Large Language Models (LLMs), with their strong pattern-matching capabilities
via pretraining on vast amounts of data, stand out as a potential candidate for
surrogate modeling. However, directly prompting a pretrained language model to
produce predictions is not feasible in many scientific domains due to the
scarcity of domain-specific data in the pretraining corpora and the challenges
of articulating complex problems in natural language. In this work, we
introduce LICO, a general-purpose model that extends arbitrary base LLMs for
black-box optimization, with a particular application to the molecular domain.
To achieve this, we equip the language model with a separate embedding layer
and prediction layer, and train the model to perform in-context predictions on
a diverse set of functions defined over the domain. Once trained, LICO can
generalize to unseen molecule properties simply via in-context prompting. LICO
achieves state-of-the-art performance on PMO, a challenging molecular
optimization benchmark comprising over 20 objective functions.

通过为预先训练的语言模型添加嵌入层和预测层，我们引入了 LICO，一个扩展了任意基础 LLM 用于黑盒优化的通用模型，特别适用于分子领域。LICO 可以通过上下文提示轻松地推广到看不见的分子属性，并在包含超过 20 个客观函数的 PMO 挑战性分子优化基准测试中取得了最先进的性能。

LICO: 大型模式语言用于上下文分子优化

LICO: Large Language Models for In-Context Molecular Optimization

Large Language Models (LLMs) encounter challenges with the unique syntax of
specific domains, such as biomolecules. Existing fine-tuning or modality
alignment techniques struggle to bridge the domain knowledge gap and understand
complex molecular data, limiting LLMs' progress in specialized fields. To
overcome these limitations, we propose an expandable and adaptable
non-parametric knowledge injection framework named Domain-specific
Retrieval-Augmented Knowledge (DRAK), aimed at enhancing reasoning capabilities
in specific domains. Utilizing knowledge-aware prompts and gold label-induced
reasoning, DRAK has developed profound expertise in the molecular domain and
the capability to handle a broad spectrum of analysis tasks. We evaluated two
distinct forms of DRAK variants, proving that DRAK exceeds previous benchmarks
on six molecular tasks within the Mol-Instructions dataset. Extensive
experiments have underscored DRAK's formidable performance and its potential to
unlock molecular insights, offering a unified paradigm for LLMs to tackle
knowledge-intensive tasks in specific domains. Our code will be available soon.

通过使用领域特定的检索增强知识 (DRAK) 框架，我们在特定领域中提高了大型语言模型的推理能力，并展示了其在分子领域的出色性能和潜力，为解决特定领域中的知识密集型任务提供了统一的范式。