The task of chemical reaction predictions (CRPs) plays a pivotal role in
advancing drug discovery and material science. However, its effectiveness is
constrained by the vast and uncertain chemical reaction space and challenges in
capturing reaction selectivity, particularly due to existing methods'
limitations in exploiting the data's inherent knowledge. To address these
challenges, we introduce a data-curated self-feedback knowledge elicitation
approach. This method starts from iterative optimization of molecular
representations and facilitates the extraction of knowledge on chemical
reaction types (RTs). Then, we employ adaptive prompt learning to infuse the
prior knowledge into the large language model (LLM). As a result, we achieve
significant enhancements: a 14.2% increase in retrosynthesis prediction
accuracy, a 74.2% rise in reagent prediction accuracy, and an expansion in the
model's capability for handling multi-task chemical reactions. This research
offers a novel paradigm for knowledge elicitation in scientific research and
showcases the untapped potential of LLMs in CRPs.

为了提高化学反应预测准确性，这项研究介绍了一种数据筛选的自我反馈知识引导方法，将先前的知识融入大型语言模型，取得了显著的预测准确度提升，同时拓展了多任务化学反应的处理能力。

一种用于化学反应预测的自反馈知识获取方法

A Self-feedback Knowledge Elicitation Approach for Chemical Reaction  Predictions

We show that existing unsupervised methods on large language model (LLM)
activations do not discover knowledge -- instead they seem to discover whatever
feature of the activations is most prominent. The idea behind unsupervised
knowledge elicitation is that knowledge satisfies a consistency structure,
which can be used to discover knowledge. We first prove theoretically that
arbitrary features (not just knowledge) satisfy the consistency structure of a
particular leading unsupervised knowledge-elicitation method,
contrast-consistent search (Burns et al. - arXiv:2212.03827). We then present a
series of experiments showing settings in which unsupervised methods result in
classifiers that do not predict knowledge, but instead predict a different
prominent feature. We conclude that existing unsupervised methods for
discovering latent knowledge are insufficient, and we contribute sanity checks
to apply to evaluating future knowledge elicitation methods. Conceptually, we
hypothesise that the identification issues explored here, e.g. distinguishing a
model's knowledge from that of a simulated character's, will persist for future
unsupervised methods.

现有的无监督方法无法发现知识，而似乎只发现活跃性的主要特征。我们首先从理论上证明了任意特征（不仅仅是知识）满足一种特定的领先无监督知识引出方法的一致性结构。然后通过一系列实验展示了未监督方法在某些情况下会导致分类器预测非知识的明显特征。我们得出结论，现有的无监督方法不能充分发现潜在的知识，并提出了合理的检查方法来评估未来的知识引出方法。在概念上，我们推测这里探讨的识别问题，例如区分模型知识和模拟角色的知识，将持续存在于未来的无监督方法中。

无监督 LLM 知识发现中的挑战

Challenges with unsupervised LLM knowledge discovery

Pre-trained language models (LMs) store knowledge in their parameters and can
generate informative responses when used in conversational systems. However,
LMs suffer from the problem of "hallucination:" they may generate
plausible-looking statements that are irrelevant or factually incorrect. To
address this problem, we propose a contrastive learning scheme, named MixCL. A
novel mixed contrastive objective is proposed to explicitly optimize the
implicit knowledge elicitation process of LMs, and thus reduce their
hallucination in conversations. We also examine negative sampling strategies of
retrieved hard negatives and model-generated negatives. We conduct experiments
on Wizard-of-Wikipedia, a public, open-domain knowledge-grounded dialogue
benchmark, and assess the effectiveness of MixCL. MixCL effectively reduces the
hallucination of LMs in conversations and achieves the highest performance
among LM-based dialogue agents in terms of relevancy and factuality. We show
that MixCL achieves comparable performance to state-of-the-art KB-based
approaches while enjoying notable advantages in terms of efficiency and
scalability.

该论文提出了一种名为 MixCL 的对比学习方案，通过显式优化 LM 的内在知识引出过程以减少困扰其会话的幻觉，在 Wizard-of-Wikipedia 上的实验证实了 MixCL 在对话中有效地减少了 LM 的幻觉，并在相关性和事实准确性方面实现了 LM 对话代理的最高性能。