This paper reviews the state-of-the-art of hybrid language models
architectures and strategies for "complex" question-answering (QA, CQA, CPS).
Large Language Models (LLM) are good at leveraging public data on standard
problems but once you want to tackle more specific complex questions or
problems you may need specific architecture, knowledge, skills, methods,
sensitive data protection, explainability, human approval and versatile
feedback... We identify key elements augmenting LLM to solve complex questions
or problems. We extend findings from the robust community edited research
papers BIG, BLOOM and HELM which open source, benchmark and analyze limits and
challenges of LLM in terms of tasks complexity and strict evaluation on
accuracy (e.g. fairness, robustness, toxicity, ...). Recent projects like
ChatGPT and GALACTICA have allowed non-specialists to grasp the great potential
as well as the equally strong limitations of language models in complex QA.
Hybridizing these models with different components could allow to overcome
these different limits and go much further. We discuss some challenges
associated with complex QA, including domain adaptation, decomposition and
efficient multi-step QA, long form and non-factoid QA, safety and
multi-sensitivity data protection, multimodal search, hallucinations,
explainability and truthfulness, temproal reasoning. Therefore, we analyze
current solutions and promising research trends, using elements such as: hybrid
LLM architectures, active human reinforcement learning supervised with AI,
prompting adaptation, neuro-symbolic and structured knowledge grounding,
program synthesis, iterated decomposition and others.

本文回顾了混合语言模型在复杂问答（QA，CQA，CPS）中架构和策略的最新进展，指出了提高 LLM 在处理背景知识、安全数据保护、解释性等方面的方法，并探讨了与复杂 QA 相关的挑战及当前解决方案和发展趋势。