Automatic summarization of mass-emergency events plays a critical role in
disaster management. The second edition of CrisisFACTS aims to advance disaster
summarization based on multi-stream fact-finding with a focus on web sources
such as Twitter, Reddit, Facebook, and Webnews. Here, participants are asked to
develop systems that can extract key facts from several disaster-related
events, which ultimately serve as a summary. This paper describes our method to
tackle this challenging task. We follow previous work and propose to use a
combination of retrieval, reranking, and an embarrassingly simple
instruction-following summarization. The two-stage retrieval pipeline relies on
BM25 and MonoT5, while the summarizer module is based on the open-source Large
Language Model (LLM) LLaMA-13b. For summarization, we explore a Question
Answering (QA)-motivated prompting approach and find the evidence useful for
extracting query-relevant facts. The automatic metrics and human evaluation
show strong results but also highlight the gap between open-source and
proprietary systems.

自动总结大规模突发事件在灾难管理中起着关键作用。CrisisFACTS 的第二版旨在基于多流事实查证，重点关注 Twitter、Reddit、Facebook 和 Webnews 等网络信息源的灾难总结。本文描述了我们解决这个具有挑战性任务的方法。我们依照以前的工作，并提出使用检索、重新排序和简单指令遵循总结相结合的方法。两阶段检索管道依赖于 BM25 和 MonoT5，而总结模块基于开源大型语言模型 LLaMA-13b。对于总结，我们采用了一个问题回答 (QA) 驱动的提示方法，并发现证据对于提取与查询相关的事实是有用的。自动评估和人工评估结果表明具有很强的效果，但也凸显了开源和专有系统之间的差距。