This paper proposes new framework of communication system leveraging
promising generation capabilities of multi-modal generative models. Regarding
nowadays smart applications, successful communication can be made by conveying
the perceptual meaning, which we set as text prompt. Text serves as a suitable
semantic representation of image data as it has evolved to instruct an image or
generate image through multi-modal techniques, by being interpreted in a manner
similar to human cognition. Utilizing text can also reduce the overload
compared to transmitting the intact data itself. The transmitter converts
objective image to text through multi-model generation process and the receiver
reconstructs the image using reverse process. Each word in the text sentence
has each syntactic role, responsible for particular piece of information the
text contains. For further efficiency in communication load, the transmitter
sequentially sends words in priority of carrying the most information until
reaches successful communication. Therefore, our primary focus is on the
promising design of a communication system based on image-to-text
transformation and the proposed schemes for sequentially transmitting word
tokens. Our work is expected to pave a new road of utilizing state-of-the-art
generative models to real communication systems

该论文提出了一种新的通信系统框架，利用多模式生成模型的优势生成有前途的代沟能力。我们的主要研究方向是基于图像到文本转换和顺序传输单词令牌的通信系统设计，旨在为实际通信系统利用最先进的生成模型铺平新的道路。

逐步文本到图像生成的序贯语义生成通信

Sequential Semantic Generative Communication for Progressive  Text-to-Image Generation

Finding word boundaries in continuous speech is challenging as there is
little or no equivalent of a 'space' delimiter between words. Popular Bayesian
non-parametric models for text segmentation use a Dirichlet process to jointly
segment sentences and build a lexicon of word types. We introduce DP-Parse,
which uses similar principles but only relies on an instance lexicon of word
tokens, avoiding the clustering errors that arise with a lexicon of word types.
On the Zero Resource Speech Benchmark 2017, our model sets a new speech
segmentation state-of-the-art in 5 languages. The algorithm monotonically
improves with better input representations, achieving yet higher scores when
fed with weakly supervised inputs. Despite lacking a type lexicon, DP-Parse can
be pipelined to a language model and learn semantic and syntactic
representations as assessed by a new spoken word embedding benchmark.

提出 DP-Parse 模型，使用基于实例词汇表的 Dirichlet 过程对语音信号进行分割处理，并且可以与语言模型配合使用，通过新的口语词嵌入数据集对其语义和句法表征能力做出评估。