Advanced artificial intelligence (AI) systems with access to millions of
research papers could inspire new research ideas that may not be conceived by
humans alone. However, how interesting are these AI-generated ideas, and how
can we improve their quality? Here, we introduce SciMuse, a system that uses an
evolving knowledge graph built from more than 58 million scientific papers to
generate personalized research ideas via an interface to GPT-4. We conducted a
large-scale human evaluation with over 100 research group leaders from the Max
Planck Society, who ranked more than 4,000 personalized research ideas based on
their level of interest. This evaluation allows us to understand the
relationships between scientific interest and the core properties of the
knowledge graph. We find that data-efficient machine learning can predict
research interest with high precision, allowing us to optimize the
interest-level of generated research ideas. This work represents a step towards
an artificial scientific muse that could catalyze unforeseen collaborations and
suggest interesting avenues for scientists.

使用超过 58 百万科学论文构建的知识图谱，并结合 GPT-4 接口生成个性化研究想法的 SciMuse 系统，通过与马克斯・普朗克研究所的 100 多位研究团队领导的大规模人工评估，我们发现数据高效的机器学习可以高精度预测研究兴趣，从而优化产生的研究想法的兴趣水平。这项工作是朝着一种人工科学启发的方向迈进，可以催生意料之外的合作，并为科学家提供有趣的研究方向。

利用知识图谱和大型语言模型生成和人工专家评估有趣的研究思路

Generation and human-expert evaluation of interesting research ideas  using knowledge graphs and large language models

In the last decade, the Winograd Schema Challenge (WSC) has become a central
aspect of the research community as a novel litmus test. Consequently, the WSC
has spurred research interest because it can be seen as the means to understand
human behavior. In this regard, the development of new techniques has made
possible the usage of Winograd schemas in various fields, such as the design of
novel forms of CAPTCHAs.
Work from the literature that established a baseline for human adult
performance on the WSC has shown that not all schemas are the same, meaning
that they could potentially be categorized according to their perceived
hardness for humans. In this regard, this \textit{hardness-metric} could be
used in future challenges or in the WSC CAPTCHA service to differentiate
between Winograd schemas.
Recent work of ours has shown that this could be achieved via the design of
an automated system that is able to output the hardness-indexes of Winograd
schemas, albeit with limitations regarding the number of schemas it could be
applied on. This paper adds to previous research by presenting a new system
that is based on Machine Learning (ML), able to output the hardness of any
Winograd schema faster and more accurately than any other previously used
method. Our developed system, which works within two different approaches,
namely the random forest and deep learning (LSTM-based), is ready to be used as
an extension of any other system that aims to differentiate between Winograd
schemas, according to their perceived hardness for humans. At the same time,
along with our developed system we extend previous work by presenting the
results of a large-scale experiment that shows how human performance varies
across Winograd schemas.

通过机器学习和大规模实验，本研究论文提出了一种新的系统，能够更快、更准确地输出 Winograd schemas 的难度，并扩展了已有研究，展示了人类在不同 Winograd schemas 上的表现变化。

经验与预测：一种新的酸碱试纸测试难度指标

Experience and Prediction: A Metric of Hardness for a Novel Litmus Test

This is a very short paper that briefly discusses some of the tasks that NLG
systems perform. It is of no research interest, but I have occasionally found
it useful as a way of introducing NLG to potential project collaborators who
know nothing about the field.

研究人工智能自然语言生成系统 (NLG)，探讨其任务与应用，介绍 NLG 领域基本知识给潜在的合作伙伴。