Large Language Models (LLMs) have been applied to build several automation
and personalized question-answering prototypes so far. However, scaling such
prototypes to robust products with minimized hallucinations or fake responses
still remains an open challenge, especially in niche data-table heavy domains
such as financial decision making. In this work, we present a novel
Langchain-based framework that transforms data tables into hierarchical textual
data chunks to enable a wide variety of actionable question answering. First,
the user-queries are classified by intention followed by automated retrieval of
the most relevant data chunks to generate customized LLM prompts per query.
Next, the custom prompts and their responses undergo multi-metric scoring to
assess for hallucinations and response confidence. The proposed system is
optimized with user-query intention classification, advanced prompting, data
scaling capabilities and it achieves over 90% confidence scores for a variety
of user-queries responses ranging from {What, Where, Why, How, predict, trend,
anomalies, exceptions} that are crucial for financial decision making
applications. The proposed data to answers framework can be extended to other
analytical domains such as sales and payroll to ensure optimal hallucination
control guardrails.

通过利用 Langchain 框架将数据表转化为分层文本数据块，本研究提出了一种新颖的方法来解决数据表重型领域（如金融决策）中的自动问答问题，实现针对各种用户查询生成定制化的语言模型提示，并在幻觉和响应置信度方面进行了多度量标评估。该系统在用户查询响应方面取得了超过 90% 的置信度，并可应用于其他分析领域以确保最佳幻觉控制保障。