By integrating recent advances in large language models (LLMs) and generative
models into the emerging semantic communication (SC) paradigm, in this article
we put forward to a novel framework of language-oriented semantic communication
(LSC). In LSC, machines communicate using human language messages that can be
interpreted and manipulated via natural language processing (NLP) techniques
for SC efficiency. To demonstrate LSC's potential, we introduce three
innovative algorithms: 1) semantic source coding (SSC) which compresses a text
prompt into its key head words capturing the prompt's syntactic essence while
maintaining their appearance order to keep the prompt's context; 2) semantic
channel coding (SCC) that improves robustness against errors by substituting
head words with their lenghthier synonyms; and 3) semantic knowledge
distillation (SKD) that produces listener-customized prompts via in-context
learning the listener's language style. In a communication task for progressive
text-to-image generation, the proposed methods achieve higher perceptual
similarities with fewer transmissions while enhancing robustness in noisy
communication channels.

将大型语言模型和生成模型整合到新兴的语义交流模式中，我们提出了一种新的面向语言的语义交流框架（LSC）。在 LSC 中，机器使用人类语言消息进行交流，可以通过自然语言处理技术进行解释和操作以提高语义交流效率。通过引入三种创新算法：1）语义源编码（SSC）将文本提示压缩为捕捉提示的句法本质的关键词，同时保持它们的出现顺序以保持提示的上下文；2）语义通道编码（SCC）通过用更长的同义词替换关键词来提高对错误的鲁棒性；3）语义知识蒸馏（SKD）通过在上下文中学习听者的语言风格生成定制的提示。在一项渐进文本到图像生成的通信任务中，所提出的方法在减少传输次数的同时实现更高的感知相似度，并提高在嘈杂的通信信道中的鲁棒性。

面向语言的通信：文本到图像生成的语义编码和知识蒸馏

Language-Oriented Communication with Semantic Coding and Knowledge  Distillation for Text-to-Image Generation

Research in multilingual speech-to-text translation is topical. Having a
single model that supports multiple translation tasks is desirable. The goal of
this work it to improve cross-lingual transfer learning in multilingual
speech-to-text translation via semantic knowledge distillation. We show that by
initializing the encoder of the encoder-decoder sequence-to-sequence
translation model with SAMU-XLS-R, a multilingual speech transformer encoder
trained using multi-modal (speech-text) semantic knowledge distillation, we
achieve significantly better cross-lingual task knowledge transfer than the
baseline XLS-R, a multilingual speech transformer encoder trained via
self-supervised learning. We demonstrate the effectiveness of our approach on
two popular datasets, namely, CoVoST-2 and Europarl. On the 21 translation
tasks of the CoVoST-2 benchmark, we achieve an average improvement of 12.8 BLEU
points over the baselines. In the zero-shot translation scenario, we achieve an
average gain of 18.8 and 11.9 average BLEU points on unseen medium and
low-resource languages. We make similar observations on Europarl speech
translation benchmark.

本论文旨在通过语义知识蒸馏来改进多语言语音转文本翻译中的跨语言迁移学习。通过使用 SAMU-XLS-R 来初始化编码器，我们在 CoVoST-2 和 Europarl 语音转文本数据集上实现了比基线模型更好的交叉语言任务知识传输，其中平均提高了 12.8 个 BLEU 分数。在零 - shot 翻译情景下，我们在未见过的中低资源语言上平均提高了 18.8 分和 11.9 个 BLEU 分数。