Regenerating natural language explanations in the scientific domain has been
proposed as a benchmark to evaluate complex multi-hop and explainable
inference. In this context, large language models can achieve state-of-the-art
performance when employed as cross-encoder architectures and fine-tuned on
human-annotated explanations. However, while much attention has been devoted to
the quality of the explanations, the problem of performing inference
efficiently is largely under-studied. Cross-encoders, in fact, are
intrinsically not scalable, possessing limited applicability to real-world
scenarios that require inference on massive facts banks. To enable complex
multi-hop reasoning at scale, this paper focuses on bi-encoder architectures,
investigating the problem of scientific explanation regeneration at the
intersection of dense and sparse models. Specifically, we present SCAR (for
Scalable Autoregressive Inference), a hybrid framework that iteratively
combines a Transformer-based bi-encoder with a sparse model of explanatory
power, designed to leverage explicit inference patterns in the explanations.
Our experiments demonstrate that the hybrid framework significantly outperforms
previous sparse models, achieving performance comparable with that of
state-of-the-art cross-encoders while being approx 50 times faster and scalable
to corpora of millions of facts. Further analyses on semantic drift and
multi-hop question answering reveal that the proposed hybridisation boosts the
quality of the most challenging explanations, contributing to improved
performance on downstream inference tasks.

研究了在自然语言解释中使用双编码器模型进行科学推理，提出了一个名为 SCAR 的混合框架，该框架结合了基于变压器的双编码器和稀疏模型，能够在大规模事实库上实现复杂的多跳推理，并提高了下游推理任务的性能表现。