The distributed (federated) LLM is an important method for co-training the
domain-specific LLM using siloed data. However, maliciously stealing model
parameters and data from the server or client side has become an urgent problem
to be solved. In this paper, we propose a secure distributed LLM based on model
slicing. In this case, we deploy the Trusted Execution Environment (TEE) on
both the client and server side, and put the fine-tuned structure (LoRA or
embedding of P-tuning v2) into the TEE. Then, secure communication is executed
in the TEE and general environments through lightweight encryption. In order to
further reduce the equipment cost as well as increase the model performance and
accuracy, we propose a split fine-tuning scheme. In particular, we split the
LLM by layers and place the latter layers in a server-side TEE (the client does
not need a TEE). We then combine the proposed Sparsification Parameter
Fine-tuning (SPF) with the LoRA part to improve the accuracy of the downstream
task. Numerous experiments have shown that our method guarantees accuracy while
maintaining security.

提出了一种基于模型切片的安全分布式 LLM 方法，使用可信执行环境（TEE）在客户端和服务器端部署，并通过轻量级加密在 TEE 和一般环境中执行安全通信，进一步降低设备成本同时提高模型性能和准确性。

大型语言模型的快速、高效、安全的分布式训练框架

A Fast, Performant, Secure Distributed Training Framework For Large  Language Model

In this paper, a generalization of deep learning-aided joint source channel
coding (Deep-JSCC) approach to secure communications is studied. We propose an
end-to-end (E2E) learning-based approach for secure communication against
multiple eavesdroppers over complex-valued fading channels. Both scenarios of
colluding and non-colluding eavesdroppers are studied. For the colluding
strategy, eavesdroppers share their logits to collaboratively infer private
attributes based on ensemble learning method, while for the non-colluding setup
they act alone. The goal is to prevent eavesdroppers from inferring private
(sensitive) information about the transmitted images, while delivering the
images to a legitimate receiver with minimum distortion. By generalizing the
ideas of privacy funnel and wiretap channel coding, the trade-off between the
image recovery at the legitimate node and the information leakage to the
eavesdroppers is characterized. To solve this secrecy funnel framework, we
implement deep neural networks (DNNs) to realize a data-driven secure
communication scheme, without relying on a specific data distribution.
Simulations over CIFAR-10 dataset verifies the secrecy-utility trade-off.
Adversarial accuracy of eavesdroppers are also studied over Rayleigh fading,
Nakagami-m, and AWGN channels to verify the generalization of the proposed
scheme. Our experiments show that employing the proposed secure neural encoding
can decrease the adversarial accuracy by 28%.

本文研究了一种基于深度学习辅助的联合源信道编码（Deep-JSCC）方法的推广，用于安全通信。我们提出了一种端到端（E2E）学习的方法，用于抵御复杂衰落信道上多个窃听者的攻击。通过推广隐私漏斗和窃听信道编码的思想，我们揭示了合法接收节点的图像恢复和对窃听者信息泄漏之间的权衡关系。通过实验验证了所提出方案的泄密性能和实用性能，并研究了在不同信道条件下窃听者的敌对准确性。实验证明所提出的安全神经编码方法可以将敌对准确性降低 28％。

多窃听者下的安全深度 JSCC

Secure Deep-JSCC Against Multiple Eavesdroppers

In this paper, we study an intelligent reflecting surface (IRS)-aided
wireless secure communication system for physical layer security, where an IRS
is deployed to adjust its surface reflecting elements to guarantee secure
communication of multiple legitimate users in the presence of multiple
eavesdroppers. Aiming to improve the system secrecy rate, a design problem for
jointly optimizing the base station (BS)'s beamforming and the IRS's reflecting
beamforming is formulated given the different quality of service (QoS)
requirements and time-varying channel condition. As the system is highly
dynamic and complex, and it is challenging to address the non-convex
optimization problem, a novel deep reinforcement learning (DRL)-based secure
beamforming approach is firstly proposed to achieve the optimal beamforming
policy against eavesdroppers in dynamic environments. Furthermore,
post-decision state (PDS) and prioritized experience replay (PER) schemes are
utilized to enhance the learning efficiency and secrecy performance.
Specifically, PDS is capable of tracing the environment dynamic characteristics
and adjust the beamforming policy accordingly. Simulation results demonstrate
that the proposed deep PDS-PER learning-based secure beamforming approach can
significantly improve the system secrecy rate and QoS satisfaction probability
in IRS-aided secure communication systems.

利用智能反射表面和深度强化学习技术优化基站波束和反射波束以提高系统保密率和服务质量，在不同质量服务需求和时变信道的情况下能显著提高在多个窃听器存在时的多个合法用户之间的安全通信系统的保密率和服务质量。