This paper presents a novel approach for multimodal data fusion based on the Vector-Quantized Variational Autoencoder (VQVAE) architecture. The proposed method is simple yet effective in achieving excellent reconstruction performance on paired MNIST-SVHN data and WiFi spectrogram data. Additionally, the multimodal VQVAE model is extended to the 5G communication scenario, where an end-to-end Channel State Information (CSI) feedback system is implemented to compress data transmitted between the base-station (eNodeB) and User Equipment (UE), without significant loss of performance. The proposed model learns a discriminative compressed feature space for various types of input data (CSI, spectrograms, natural images, etc), making it a suitable solution for applications with limited computational resources.

本文提出了一种基于向量量化变分自编码器架构的多模态数据融合新方法，可以在MNIST-SVHN数据和WiFi光谱数据之间以及5G通信场景下实现卓越的重构性能，并通过端到端CSI反馈系统来压缩基站（eNodeB）和用户设备（UE）之间传输的数据，为各种类型的输入数据（CSI，光谱图，自然图像等）学习一个具有区分性的压缩特征空间，是针对计算资源有限的应用的一个合适的解决方案。

无线通信和传感器网络中多模态数据融合的优化