Large Language Models (LLMs) have gained prominence in the field of Legal
Intelligence, offering potential applications in assisting legal professionals
and laymen. However, the centralized training of these Legal LLMs raises data
privacy concerns, as legal data is distributed among various institutions
containing sensitive individual information. This paper addresses this
challenge by exploring the integration of Legal LLMs with Federated Learning
(FL) methodologies. By employing FL, Legal LLMs can be fine-tuned locally on
devices or clients, and their parameters are aggregated and distributed on a
central server, ensuring data privacy without directly sharing raw data.
However, computation and communication overheads hinder the full fine-tuning of
LLMs under the FL setting. Moreover, the distribution shift of legal data
reduces the effectiveness of FL methods. To this end, in this paper, we propose
the first Federated Legal Large Language Model (FedJudge) framework, which
fine-tunes Legal LLMs efficiently and effectively. Specifically, FedJudge
utilizes parameter-efficient fine-tuning methods to update only a few
additional parameters during the FL training. Besides, we explore the continual
learning methods to preserve the global model's important parameters when
training local clients to mitigate the problem of data shifts. Extensive
experimental results on three real-world datasets clearly validate the
effectiveness of FedJudge. Code is released at
this https URL

通过联邦学习方法，本文提出了第一个高效、有效的联邦法律大型语言模型（FedJudge）框架，用于在保护数据隐私的前提下，实现对法律大型语言模型的本地调优。

FedJudge: 联合法律大型语言模型

FedJudge: Federated Legal Large Language Model

The advancement of Internet and Communication Technologies (ICTs) has led to
the era of Industry 4.0. This shift is followed by healthcare industries
creating the term Healthcare 4.0. In Healthcare 4.0, the use of IoT-enabled
medical imaging devices for early disease detection has enabled medical
practitioners to increase healthcare institutions' quality of service. However,
Healthcare 4.0 is still lagging in Artificial Intelligence and big data
compared to other Industry 4.0 due to data privacy concerns. In addition,
institutions' diverse storage and computing capabilities restrict institutions
from incorporating the same training model structure. This paper presents a
secure multi-party computation-based ensemble federated learning with
blockchain that enables heterogeneous models to collaboratively learn from
healthcare institutions' data without violating users' privacy. Blockchain
properties also allow the party to enjoy data integrity without trust in a
centralized server while also providing each healthcare institution with
auditability and version control capability.

本研究提出一种基于多方计算的安全集成联邦学习与区块链的方法，使异构模型能够在保护用户隐私的前提下协同学习医疗机构的数据，并利用区块链的属性实现数据完整性和审计控制能力。