For reliable transmission across a noisy communication channel, classical
results from information theory show that it is asymptotically optimal to
separate out the source and channel coding processes. However, this
decomposition can fall short in the finite bit-length regime, as it requires
non-trivial tuning of hand-crafted codes and assumes infinite computational
power for decoding. In this work, we propose to jointly learn the encoding and
decoding processes using a new discrete variational autoencoder model. By
adding noise into the latent codes to simulate the channel during training, we
learn to both compress and error-correct given a fixed bit-length and
computational budget. We obtain codes that are not only competitive against
several separation schemes, but also learn useful robust representations of the
data for downstream tasks such as classification. Finally, inference
amortization yields an extremely fast neural decoder, almost an order of
magnitude faster compared to standard decoding methods based on iterative
belief propagation.

本文提出了一种用于编解码的离散自编码器模型，通过添加噪声来模拟信道以实现数据压缩和容错，得到了与现有方案相当的编码标准，并学习了下游任务（如分类）的有用鲁棒表征。最后，通过推理摊销，实现了一个快速的神经解码器，与迭代置信传播方法相比，速度提高了近一个数量级。