This paper presents a novel approach for multimodal data fusion based on the
Vector-Quantized Variational Autoencoder (VQVAE) architecture. The proposed
method is simple yet effective in achieving excellent reconstruction
performance on paired MNIST-SVHN data and WiFi spectrogram data. Additionally,
the multimodal VQVAE model is extended to the 5G communication scenario, where
an end-to-end Channel State Information (CSI) feedback system is implemented to
compress data transmitted between the base-station (eNodeB) and User Equipment
(UE), without significant loss of performance. The proposed model learns a
discriminative compressed feature space for various types of input data (CSI,
spectrograms, natural images, etc), making it a suitable solution for
applications with limited computational resources.

本文提出了一种基于向量量化变分自编码器架构的多模态数据融合新方法，可以在 MNIST-SVHN 数据和 WiFi 光谱数据之间以及 5G 通信场景下实现卓越的重构性能，并通过端到端 CSI 反馈系统来压缩基站（eNodeB）和用户设备（UE）之间传输的数据，为各种类型的输入数据（CSI，光谱图，自然图像等）学习一个具有区分性的压缩特征空间，是针对计算资源有限的应用的一个合适的解决方案。

无线通信和传感器网络中多模态数据融合的优化

Streamlining Multimodal Data Fusion in Wireless Communication and Sensor Networks

Speech-to-text translation has many potential applications for low-resource
languages, but the typical approach of cascading speech recognition with
machine translation is often impossible, since the transcripts needed to train
a speech recognizer are usually not available for low-resource languages.
Recent work has found that neural encoder-decoder models can learn to directly
translate foreign speech in high-resource scenarios, without the need for
intermediate transcription. We investigate whether this approach also works in
settings where both data and computation are limited. To make the approach
efficient, we make several architectural changes, including a change from
character-level to word-level decoding. We find that this choice yields crucial
speed improvements that allow us to train with fewer computational resources,
yet still performs well on frequent words. We explore models trained on between
20 and 160 hours of data, and find that although models trained on less data
have considerably lower BLEU scores, they can still predict words with
relatively high precision and recall---around 50% for a model trained on 50
hours of data, versus around 60% for the full 160 hour model. Thus, they may
still be useful for some low-resource scenarios.

通过实验研究，我们发现在资源有限的情况下，从字符级识别转化成单词级可以显著提高低资源语言语音翻译的速度，并且在训练数据相对较少的情况下，仍然可以实现相对较高的词汇精度和召回率。