Spoken language understanding (SLU), one of the key enabling technologies for
human-computer interaction in IoT devices, provides an easy-to-use user
interface. Human speech can contain a lot of user-sensitive information, such
as gender, identity, and sensitive content. New types of security and privacy
breaches have thus emerged. Users do not want to expose their personal
sensitive information to malicious attacks by untrusted third parties. Thus,
the SLU system needs to ensure that a potential malicious attacker cannot
deduce the sensitive attributes of the users, while it should avoid greatly
compromising the SLU accuracy. To address the above challenge, this paper
proposes a novel SLU multi-task privacy-preserving model to prevent both the
speech recognition (ASR) and identity recognition (IR) attacks. The model uses
the hidden layer separation technique so that SLU information is distributed
only in a specific portion of the hidden layer, and the other two types of
information are removed to obtain a privacy-secure hidden layer. In order to
achieve good balance between efficiency and privacy, we introduce a new
mechanism of model pre-training, namely joint adversarial training, to further
enhance the user privacy. Experiments over two SLU datasets show that the
proposed method can reduce the accuracy of both the ASR and IR attacks close to
that of a random guess, while leaving the SLU performance largely unaffected.

本研究提出了一种新颖的跨任务保护用户隐私的口语理解模型，通过利用隐藏层分离技术，将用户信息仅分布在特定部分的隐藏层中并移除其他类型信息，从而实现隐私安全的隐藏层。为了在效率和隐私之间取得良好的平衡，引入了一种新的模型预训练机制，即联合对抗训练，以进一步增强用户隐私。实验证明，该方法能将语音识别和身份识别攻击的准确性降低至随机猜测水平，同时对口语理解性能影响较小。

隐私保护的端到端口语言理解

Privacy-Preserving End-to-End Spoken Language Understanding

Vertical Federated Learning (VFL) attracts increasing attention because it
empowers multiple parties to jointly train a privacy-preserving model over
vertically partitioned data. Recent research has shown that applying
zeroth-order optimization (ZOO) has many advantages in building a practical VFL
algorithm. However, a vital problem with the ZOO-based VFL is its slow
convergence rate, which limits its application in handling modern large models.
To address this problem, we propose a cascaded hybrid optimization method in
VFL. In this method, the downstream models (clients) are trained with ZOO to
protect privacy and ensure that no internal information is shared. Meanwhile,
the upstream model (server) is updated with first-order optimization (FOO)
locally, which significantly improves the convergence rate, making it feasible
to train the large models without compromising privacy and security. We
theoretically prove that our VFL framework converges faster than the ZOO-based
VFL, as the convergence of our framework is not limited by the size of the
server model, making it effective for training large models with the major part
on the server. Extensive experiments demonstrate that our method achieves
faster convergence than the ZOO-based VFL framework, while maintaining an
equivalent level of privacy protection. Moreover, we show that the convergence
of our VFL is comparable to the unsafe FOO-based VFL baseline. Additionally, we
demonstrate that our method makes the training of a large model feasible.

本文提出了一个级联混合优化方法来加速零阶优化算法，从而解决基于零阶优化的垂直联邦学习算法的收敛速度慢的问题。该算法在较大模型的训练中非常有效，且具有相当的隐私保护水平。