This study delves into the capabilities and limitations of Large Language
Models (LLMs) in the challenging domain of conditional question-answering.
Utilizing the Conditional Question Answering (CQA) dataset and focusing on
generative models like T5 and UL2, we assess the performance of LLMs across
diverse question types. Our findings reveal that fine-tuned LLMs can surpass
the state-of-the-art (SOTA) performance in some cases, even without fully
encoding all input context, with an increase of 7-8 points in Exact Match (EM)
and F1 scores for Yes/No questions. However, these models encounter challenges
in extractive question answering, where they lag behind the SOTA by over 10
points, and in mitigating the risk of injecting false information. A study with
oracle-retrievers emphasizes the critical role of effective evidence retrieval,
underscoring the necessity for advanced solutions in this area. Furthermore, we
highlight the significant influence of evaluation metrics on performance
assessments and advocate for a more comprehensive evaluation framework. The
complexity of the task, the observed performance discrepancies, and the need
for effective evidence retrieval underline the ongoing challenges in this field
and underscore the need for future work focusing on refining training tasks and
exploring prompt-based techniques to enhance LLM performance in conditional
question-answering tasks.

该研究探讨了大型语言模型 (LLMs) 在具有挑战性的条件问答领域中的能力和局限性。利用条件问答 (CQA) 数据集，重点关注 T5 和 UL2 等生成模型，我们评估了 LLMs 在不同问题类型上的性能。研究发现，经过微调的 LLMs 在某些情况下可以超越现有技术在一些方面的表现，即使没有完全编码所有输入上下文，对于是 / 否问题的精确匹配 (EM) 和 F1 分数有 7-8 个点的增加。然而，这些模型在抽取性问答方面遇到了挑战，在与现有技术相比落后于 10 个以上的点，并且在减少注入错误信息的风险方面也存在问题。与神谕检索器进行的一项研究强调了有效证据检索的关键作用，强调了该领域需要先进解决方案的必要性。此外，我们强调了评估评价指标对性能评估的重要影响，并倡导使用更全面的评估框架。任务的复杂性、观察到的性能差异以及在条件问答任务中改进训练任务和探索基于提示的技术以提高 LLMs 性能的未来工作的需求，突显了这一领域面临的持续挑战。