In Greek mythology, Pistis symbolized good faith, trust, and reliability,
echoing the core principles of RAG in LLM systems. Pistis-RAG, a scalable
multi-stage framework, effectively addresses the challenges of large-scale
retrieval-augmented generation (RAG). Each stage plays a distinct role:
matching refines the search space, pre-ranking prioritizes semantically
relevant documents, and ranking aligns with the large language model's (LLM)
preferences. The reasoning and aggregating stage supports the implementation of
complex chain-of-thought (CoT) methods within this cascading structure. We
argue that the lack of strong alignment between LLMs and the external knowledge
ranking methods used in RAG tasks is relevant to the reliance on the
model-centric paradigm in RAG frameworks. A content-centric approach would
prioritize seamless integration between the LLMs and external information
sources, optimizing the content transformation process for each specific task.
Critically, our ranking stage deviates from traditional RAG approaches by
recognizing that semantic relevance alone may not directly translate to
improved generation. This is due to the sensitivity of the few-shot prompt
order, as highlighted in prior work \cite{lu2021fantastically}. Current RAG
frameworks fail to account for this crucial factor. We introduce a novel
ranking stage specifically designed for RAG systems. It adheres to information
retrieval principles while considering the unique business scenario captured by
LLM preferences and user feedback. Our approach integrates in-context learning
(ICL) methods and reasoning steps to incorporate user feedback, ensuring
efficient alignment. Experiments on the MMLU benchmark demonstrate a 9.3\%
performance improvement. The model and code will be open-sourced on GitHub.
Experiments on real-world, large-scale data validate our framework's
scalability.

使用 Pistis-RAG 框架以及内容为中心的方法，通过改进排序阶段，提高了检索增强生成系统的性能，并且能够更好地对鲜为人知的 UI 交互进行建模以及处理。

Pistis-RAG: 一种可扩展的层级框架用于可信的检索增强生成

Pistis-RAG: A Scalable Cascading Framework Towards Trustworthy  Retrieval-Augmented Generation

As large language models (LLMs) become more deeply integrated into various
sectors, understanding how they make moral judgments has become crucial,
particularly in the realm of autonomous driving. This study utilized the Moral
Machine framework to investigate the ethical decision-making tendencies of
prominent LLMs, including GPT-3.5, GPT-4, PaLM 2, and Llama 2, comparing their
responses to human preferences. While LLMs' and humans' preferences such as
prioritizing humans over pets and favoring saving more lives are broadly
aligned, PaLM 2 and Llama 2, especially, evidence distinct deviations.
Additionally, despite the qualitative similarities between the LLM and human
preferences, there are significant quantitative disparities, suggesting that
LLMs might lean toward more uncompromising decisions, compared to the milder
inclinations of humans. These insights elucidate the ethical frameworks of LLMs
and their potential implications for autonomous driving.

当 LML（大型语言模型）在各个领域中深入应用时，了解它们如何做出道德判断变得至关重要，特别是在自动驾驶领域。本研究利用 Moral Machine 框架，调查了几个重要 LML（包括 GPT-3.5，GPT-4，PaLM 2 和 Llama 2）的道德决策倾向，并将它们与人类偏好进行比较。虽然 LML 和人类的偏好在许多方面保持一致，如优先考虑人类而非宠物，倾向于拯救更多生命，但尤其是 PaLM 2 和 Llama 2 表现出明显的差异。此外，尽管 LML 和人类的偏好在质的方面相似，但在量的方面存在显著差异，这表明相较于人类的温和偏向，LML 可能更倾向于更坚决的决策。这些发现揭示了 LML 的道德框架，以及它们对自动驾驶的潜在影响。