Despite the dramatic progress in Large Language Model (LLM) development, LLMs
often provide seemingly plausible but not factual information, often referred
to as hallucinations. Retrieval-augmented LLMs provide a non-parametric
approach to solve these issues by retrieving relevant information from external
data sources and augment the training process. These models help to trace
evidence from an externally provided knowledge base allowing the model
predictions to be better interpreted and verified. In this work, we critically
evaluate these models in their ability to perform in scientific document
reasoning tasks. To this end, we tuned multiple such model variants with
science-focused instructions and evaluated them on a scientific document
reasoning benchmark for the usefulness of the retrieved document passages. Our
findings suggest that models justify predictions in science tasks with
fabricated evidence and leveraging scientific corpus as pretraining data does
not alleviate the risk of evidence fabrication.

本研究通过关键字检索对多种大型语言模型进行评估，发现这些模型在科学文档推理任务中会使用编造的证据来支持预测，利用科学语料库进行预训练无法减轻证据捏造的风险。