Large language models (LLMs) with residual augmented-generation (RAG) have
been the optimal choice for scalable generative AI solutions in the recent
past. However, the choice of use-cases that incorporate RAG with LLMs have been
either generic or extremely domain specific, thereby questioning the
scalability and generalizability of RAG-LLM approaches. In this work, we
propose a unique LLM-based system where multiple LLMs can be invoked to enable
data authentication, user query routing, data retrieval and custom prompting
for question answering capabilities from data tables that are highly varying
and large in size. Our system is tuned to extract information from
Enterprise-level data products and furnish real time responses under 10
seconds. One prompt manages user-to-data authentication followed by three
prompts to route, fetch data and generate a customizable prompt natural
language responses. Additionally, we propose a five metric scoring module that
detects and reports hallucinations in the LLM responses. Our proposed system
and scoring metrics achieve >90% confidence scores across hundreds of user
queries in the sustainability, financial health and social media domains.
Extensions to the proposed extreme RAG architectures can enable heterogeneous
source querying using LLMs.

我们提出了一种基于大型语言模型的系统，可以通过多个大型语言模型从各种规模大的企业级数据产品中提取信息，并在 10 秒内提供实时响应，同时提出了一个得分模块来检测和报告大型语言模型响应中的幻觉。在可持续发展、财务健康和社交媒体领域的数百个用户查询中，我们的系统和评分指标实现了超过 90% 的置信分数。