Consumer Health Queries (CHQs) in Bengali (Bangla), a low-resource language, often contain extraneous details, complicating efficient medical responses. This study investigates the zero-shot performance of nine advanced large language models (LLMs): GPT-3.5-Turbo, GPT-4, Claude-3.5-Sonnet, Llama3-70b-Instruct, Mixtral-8x22b-Instruct, Gemini-1.5-Pro, Qwen2-72b-Instruct, Gemma-2-27b, and Athene-70B, in summarizing Bangla CHQs. Using the BanglaCHQ-Summ dataset comprising 2,350 annotated query-summary pairs, we benchmarked these LLMs using ROUGE metrics against Bangla T5, a fine-tuned state-of-the-art model. Mixtral-8x22b-Instruct emerged as the top performing model in ROUGE-1 and ROUGE-L, while Bangla T5 excelled in ROUGE-2. The results demonstrate that zero-shot LLMs can rival fine-tuned models, achieving high-quality summaries even without task-specific training. This work underscores the potential of LLMs in addressing challenges in low-resource languages, providing scalable solutions for healthcare query summarization.

本研究针对孟加拉语这一低资源语言中的消费者健康查询（CHQs）所面临的冗余信息问题，探索了九种先进大型语言模型（LLMs）的零-shot 性能。研究发现，Mixtral-8x22b-Instruct 模型在 ROUGE-1 和 ROUGE-L 指标上表现最佳，表明这些大型语言模型在没有特定任务训练的情况下仍能生成高质量摘要，展示了它们在低资源语言医疗查询摘要化中的潜力。

孟加拉消费者健康查询摘要的大型语言模型性能评估