Recent work has shown that language models scaled to billions of parameters,
such as GPT-3, perform remarkably well in zero-shot and few-shot scenarios. In
this work, we experiment with zero-shot models in the legal case entailment
task of the COLIEE 2022 competition. Our experiments show that scaling the
number of parameters in a language model improves the F1 score of our previous
zero-shot result by more than 6 points, suggesting that stronger zero-shot
capability may be a characteristic of larger models, at least for this task.
Our 3B-parameter zero-shot model outperforms all models, including ensembles,
in the COLIEE 2021 test set and also achieves the best performance of a single
model in the COLIEE 2022 competition, second only to the ensemble composed of
the 3B model itself and a smaller version of the same model. Despite the
challenges posed by large language models, mainly due to latency constraints in
real-time applications, we provide a demonstration of our zero-shot monoT5-3b
model being used in production as a search engine, including for legal
documents. The code for our submission and the demo of our system are available
at this https URL and
this https URL, respectively.

本文中，我们使用了零 shot 模型在 COLIEE 2022 的法律案例蕴含任务中进行了实验，并发现语言模型参数数量的增加可以提高 F1 分数，尤其是对于该任务。我们的 3B 零 shot 模型在 COLIEE 2021 测试集中表现优异，在 COLIEE 2022 比赛中也取得了最佳表现，其次是由 3B 模型本身和较小版本的模型组成的集成模型。此外，我们还演示了零 shot monoT5-3b 模型在生产中如何作为搜索引擎使用，包括用于法律文件。