This paper studies how to effectively build meeting summarization systems for
real-world usage using large language models (LLMs). For this purpose, we
conduct an extensive evaluation and comparison of various closed-source and
open-source LLMs, namely, GPT-4, GPT- 3.5, PaLM-2, and LLaMA-2. Our findings
reveal that most closed-source LLMs are generally better in terms of
performance. However, much smaller open-source models like LLaMA- 2 (7B and
13B) could still achieve performance comparable to the large closed-source
models even in zero-shot scenarios. Considering the privacy concerns of
closed-source models for only being accessible via API, alongside the high cost
associated with using fine-tuned versions of the closed-source models, the
opensource models that can achieve competitive performance are more
advantageous for industrial use. Balancing performance with associated costs
and privacy concerns, the LLaMA-2-7B model looks more promising for industrial
usage. In sum, this paper offers practical insights on using LLMs for
real-world business meeting summarization, shedding light on the trade-offs
between performance and cost.

本研究通过评估和比较各种闭源和开源大型语言模型，探讨如何有效构建用于实际应用的会议摘要系统。研究结果表明，大多数闭源模型在性能方面更好，但即使在零 - shot 情况下，较小的开源模型如 LLaMA-2（7B 和 13B）仍可达到与大型闭源模型相当的性能。综合考虑闭源模型的隐私问题和使用经过微调的闭源模型的高成本，能够取得竞争性性能的开源模型更适合工业应用。在性能、成本和隐私问题之间取得平衡，LLaMA-2-7B 模型在工业应用中更具前景。总之，本文提供了使用大型语言模型进行实际业务会议摘要的实用见解，突显性能和成本之间的权衡。