This paper explores the applicability of sequence-to-sequence (Seq2Seq)
models based on LSTM units for Automatic Speech Recognition (ASR) task within
peer-to-peer learning environments. Leveraging two distinct peer-to-peer
learning methods, the study simulates the learning process of agents and
evaluates their performance in ASR task using two different ASR datasets. In a
centralized training setting, utilizing a scaled-down variant of the Deep
Speech 2 model, a single model achieved a Word Error Rate (WER) of 84\% when
trained on the UserLibri dataset, and 38\% when trained on the LJ Speech
dataset. Conversely, in a peer-to-peer learning scenario involving 55 agents,
the WER ranged from 87\% to 92\% for the UserLibri dataset, and from 52\% to
56\% for the LJ Speech dataset. The findings demonstrate the feasibility of
employing Seq2Seq models in decentralized settings, albeit with slightly higher
Word Error Rates (WER) compared to centralized training methods.

探索使用基于 LSTM 单元的序列到序列 (Seq2Seq) 模型在点对点学习环境下的自动语音识别 (ASR) 任务的适用性，通过两种不同的点对点学习方法模拟代理学习过程，并使用两个不同的 ASR 数据集评估其性能。研究发现在集中式训练环境中，使用缩小版 Deep Speech 2 模型的单个模型在 UserLibri 数据集上训练时，识别误差率 (Word Error Rate, WER) 为 84%，在 LJ Speech 数据集上训练时为 38%。然而，通过 55 个代理进行点对点学习，并使用 UserLibri 数据集和 LJ Speech 数据集进行训练时，WER 在 UserLibri 数据集上的范围为 87% 至 92%，在 LJ Speech 数据集上的范围为 52% 至 56%。结果表明，在分散式训练环境中使用 Seq2Seq 模型是可行的，尽管识别误差率 (WER) 稍高于集中式训练方法。

对等学习中的序列到序列模型：一个实用应用

Sequence-to-sequence models in peer-to-peer learning: A practical  application

Collaborative machine learning (ML) is widely used to enable institutions to
learn better models from distributed data. While collaborative approaches to
learning intuitively protect user data, they remain vulnerable to either the
server, the clients, or both, deviating from the protocol. Indeed, because the
protocol is asymmetric, a malicious server can abuse its power to reconstruct
client data points. Conversely, malicious clients can corrupt learning with
malicious updates. Thus, both clients and servers require a guarantee when the
other cannot be trusted to fully cooperate. In this work, we propose a
peer-to-peer (P2P) learning scheme that is secure against malicious servers and
robust to malicious clients. Our core contribution is a generic framework that
transforms any (compatible) algorithm for robust aggregation of model updates
to the setting where servers and clients can act maliciously. Finally, we
demonstrate the computational efficiency of our approach even with 1-million
parameter models trained by 100s of peers on standard datasets.

通过这项工作，我们提出了一种对抗恶意服务器、抵御恶意客户端的点对点学习方案，核心贡献是一个通用框架，使得兼容的模型更新聚合算法能够在服务器和客户端恶意行为的环境下运行。此外，我们展示了我们的方法在由数百个节点对标准数据集训练的百万参数模型上的计算效率。

鲁棒性和实时安全的无服务器协作学习

Robust and Actively Secure Serverless Collaborative Learning

Knowledge graph embedding plays an important role in knowledge
representation, reasoning, and data mining applications. However, for multiple
cross-domain knowledge graphs, state-of-the-art embedding models cannot make
full use of the data from different knowledge domains while preserving the
privacy of exchanged data. In addition, the centralized embedding model may not
scale to the extensive real-world knowledge graphs. Therefore, we propose a
novel decentralized scalable learning framework, \emph{Federated Knowledge
Graphs Embedding} (FKGE), where embeddings from different knowledge graphs can
be learnt in an asynchronous and peer-to-peer manner while being
privacy-preserving. FKGE exploits adversarial generation between pairs of
knowledge graphs to translate identical entities and relations of different
domains into near embedding spaces. In order to protect the privacy of the
training data, FKGE further implements a privacy-preserving neural network
structure to guarantee no raw data leakage. We conduct extensive experiments to
evaluate FKGE on 11 knowledge graphs, demonstrating a significant and
consistent improvement in model quality with at most 17.85\% and 7.90\%
increases in performance on triple classification and link prediction tasks.

本论文提出了一种名为 FKGE 的分散、可扩展、隐私保护的学习框架，可用于跨多个同一领域不同知识图谱的知识嵌入。在对 11 个不同的知识图谱进行广泛的实验后，本文表明 FKGE 的性能提高了，它的模型质量最多可增加 17.85％和 7.90％。