In this paper, we present CORE-GPT, a novel question-answering platform that
combines GPT-based language models and more than 32 million full-text open
access scientific articles from CORE. We first demonstrate that GPT3.5 and GPT4
cannot be relied upon to provide references or citations for generated text. We
then introduce CORE-GPT which delivers evidence-based answers to questions,
along with citations and links to the cited papers, greatly increasing the
trustworthiness of the answers and reducing the risk of hallucinations.
CORE-GPT's performance was evaluated on a dataset of 100 questions covering the
top 20 scientific domains in CORE, resulting in 100 answers and links to 500
relevant articles. The quality of the provided answers and and relevance of the
links were assessed by two annotators. Our results demonstrate that CORE-GPT
can produce comprehensive and trustworthy answers across the majority of
scientific domains, complete with links to genuine, relevant scientific
articles.

本研究提出了 CORE-GPT，一个结合了 GPT 语言模型和超过 3200 万篇 CORE 科学论文的问答平台。我们首先证明了 GPT3.5 和 GPT4 不能提供所生成文本的参考文献和引用。然后，我们介绍了 CORE-GPT，它能够提供有据可依的答案，并附带引文和文章链接，大大提高了答案的可信度，减少了虚构性的风险。CORE-GPT 在 CORE 数据库中涵盖了 20 个科学领域的 100 个问题上进行了性能评估，得出了 100 个答案和 500 篇相关文章的链接。两个标注者评估了提供的答案的质量和链接的相关性。我们的结果表明，CORE-GPT 能够在绝大多数科学领域生成全面且可信赖的答案，同时提供真实且相关的科学文章的链接。

CORE-GPT：结合开放获取研究和大型语言模型进行可信、可靠的问答

CORE-GPT: Combining Open Access research and large language models for  credible, trustworthy question answering

Pretrained Language Models (LMs) have demonstrated ability to perform
numerical reasoning by extrapolating from a few examples in few-shot settings.
However, the extent to which this extrapolation relies on robust reasoning is
unclear. In this paper, we investigate how well these models reason with terms
that are less frequent in the pretraining data. In particular, we examine the
correlations between the model performance on test instances and the frequency
of terms from those instances in the pretraining data. We measure the strength
of this correlation for a number of GPT-based language models (pretrained on
the Pile dataset) on various numerical deduction tasks (e.g., arithmetic and
unit conversion). Our results consistently demonstrate that models are more
accurate on instances whose terms are more prevalent, in some cases above
$70\%$ (absolute) more accurate on the top 10\% frequent terms in comparison to
the bottom 10\%. Overall, although LMs exhibit strong performance at few-shot
numerical reasoning tasks, our results raise the question of how much models
actually generalize beyond pretraining data, and we encourage researchers to
take the pretraining data into account when interpreting evaluation results.

本文研究预训练语言模型在数值推理方面的能力，以及该能力强弱与预训练数据中各项词语的出现频率之间的关系。研究结果表明，模型对出现频率较高的词语表现更为准确。此外，作者认为在解释理论评价结果时，应考虑到预训练数据带来的影响。