Large language models with billions of parameters, such as GPT-3.5, GPT-4,
and LLaMA, are increasingly prevalent. Numerous studies have explored effective
prompting techniques to harness the power of these LLMs for various research
problems. Retrieval, specifically in the legal data domain, poses a challenging
task for the direct application of Prompting techniques due to the large number
and substantial length of legal articles. This research focuses on maximizing
the potential of prompting by placing it as the final phase of the retrieval
system, preceded by the support of two phases: BM25 Pre-ranking and BERT-based
Re-ranking. Experiments on the COLIEE 2023 dataset demonstrate that integrating
prompting techniques on LLMs into the retrieval system significantly improves
retrieval accuracy. However, error analysis reveals several existing issues in
the retrieval system that still need resolution.

将提示技术作为检索系统中的最后阶段，通过 BM25 预排名和基于 BERT 的再排名的支持，最大限度地提高提示的潜力，从而显着提高了检索准确性。