Large language models (LLMs) have gained widespread interest due to their
ability to process human language and perform tasks on which they have not been
explicitly trained. This is relevant for the chemical sciences, which face the
problem of small and diverse datasets that are frequently in the form of text.
LLMs have shown promise in addressing these issues and are increasingly being
harnessed to predict chemical properties, optimize reactions, and even design
and conduct experiments autonomously. However, we still have only a very
limited systematic understanding of the chemical reasoning capabilities of
LLMs, which would be required to improve models and mitigate potential harms.
Here, we introduce "ChemBench," an automated framework designed to rigorously
evaluate the chemical knowledge and reasoning abilities of state-of-the-art
LLMs against the expertise of human chemists. We curated more than 7,000
question-answer pairs for a wide array of subfields of the chemical sciences,
evaluated leading open and closed-source LLMs, and found that the best models
outperformed the best human chemists in our study on average. The models,
however, struggle with some chemical reasoning tasks that are easy for human
experts and provide overconfident, misleading predictions, such as about
chemicals' safety profiles. These findings underscore the dual reality that,
although LLMs demonstrate remarkable proficiency in chemical tasks, further
research is critical to enhancing their safety and utility in chemical
sciences. Our findings also indicate a need for adaptations to chemistry
curricula and highlight the importance of continuing to develop evaluation
frameworks to improve safe and useful LLMs.

通过 ChemBench 等评估框架，我们发现大型语言模型在化学科学中展示出卓越的能力，但仍需进一步研究以提高其安全性和实用性。

大型语言模型是超人级化学家吗？

Are large language models superhuman chemists?

This two-part review examines how automation has contributed to different
aspects of discovery in the chemical sciences. In this second part, we reflect
on a selection of exemplary studies. It is increasingly important to articulate
what the role of automation and computation has been in the scientific process
and how that has or has not accelerated discovery. One can argue that even the
best automated systems have yet to ``discover'' despite being incredibly useful
as laboratory assistants. We must carefully consider how they have been and can
be applied to future problems of chemical discovery in order to effectively
design and interact with future autonomous platforms.
The majority of this article defines a large set of open research directions,
including improving our ability to work with complex data, build empirical
models, automate both physical and computational experiments for validation,
select experiments, and evaluate whether we are making progress toward the
ultimate goal of autonomous discovery. Addressing these practical and
methodological challenges will greatly advance the extent to which autonomous
systems can make meaningful discoveries.

这篇论文主要研究了在化学科学领域自动化技术的应用，探讨了目前自动科学发现的局限性，并提出了未来自主发现的实际应用和方法论方向。

化学科学中的自主发现第二部分：展望

Autonomous discovery in the chemical sciences part II: Outlook

The chemical sciences are producing an unprecedented amount of large,
high-dimensional data sets containing chemical structures and associated
properties. However, there are currently no algorithms to visualize such data
while preserving both global and local features with a sufficient level of
detail to allow for human inspection and interpretation. Here, we propose a
solution to this problem with a new data visualization method, TMAP, capable of
representing data sets of up to millions of data points and arbitrary high
dimensionality as a two-dimensional tree (this http URL).
Visualizations based on TMAP are better suited than t-SNE or UMAP for the
exploration and interpretation of large data sets due to their tree-like
nature, increased local and global neighborhood and structure preservation, and
the transparency of the methods the algorithm is based on. We apply TMAP to the
most used chemistry data sets including databases of molecules such as ChEMBL,
FDB17, the Natural Products Atlas, DSSTox, as well as to the MoleculeNet
benchmark collection of data sets. We also show its broad applicability with
further examples from biology, particle physics, and literature.

本文介绍了一种新的数据可视化方法 TMAP，可将高维数据集表示为二维树，具有结构保持、透明性等优点，比 t-SNE 或 UMAP 更适用于化学、生物、物理等领域的数据集的探索和解释。