With the deluge of information delivered by the daily news cycle, there is a
growing need to effectively and efficiently summarize news feeds for quick
consumption. We leverage large language models (LLMs), with their advanced
learning and generative abilities as compared to conventional language models,
to generate concise and coherent summaries for news articles from the XSum
dataset. Our paper focuses on two key aspects of LLMs: Efficient in-context
Learning (ELearn) and Parameter Efficient Fine-tuning (EFit). Under ELearn, we
find that increasing the number of shots in prompts and utilizing simple
templates generally improve the quality of summaries. We also find that
utilizing relevant examples in few-shot learning for ELearn does not improve
model performance. In addition, we studied EFit using different methods and
demonstrate that fine-tuning the first layer of LLMs produces better outcomes
as compared to fine-tuning other layers or utilizing LoRA. We also find that
leveraging more relevant training samples using selective layers does not
result in better performance. By combining ELearn and EFit, we create a new
model (ELearnFit) that leverages the benefits of both few-shot learning and
fine-tuning and produces superior performance to either model alone. We also
use ELearnFit to highlight the trade-offs between prompting and fine-tuning,
especially for situations where only a limited number of annotated samples are
available. Ultimately, our research provides practical techniques to optimize
news summarization during the prompting and fine-tuning stages and enhances the
synthesis of news articles.

通过利用大型语言模型，本文研究了新闻摘要的两个关键方面：高效的上下文学习和参数高效的微调，最终创建了一种结合了前者的利用其学习能力和后者的微调优势的新模型，以提升新闻文章的综合能力。