Compression is at the heart of effective representation learning. However, lossy compression is typically achieved through simple parametric models like Gaussian noise to preserve analytic tractability, and the limitations this imposes on learning are largely unexplored. Further, the Gaussian prior assumptions in models such as variational autoencoders (VAEs) provide only an upper bound on the compression rate in general. We introduce a new noise channel, Echo noise, that admits a simple, exact expression for mutual information for arbitrary input distributions. The noise is constructed in a data-driven fashion that does not require restrictive distributional assumptions. With its complex encoding mechanism and exact rate regularization, Echo leads to improved bounds on log-likelihood and dominates $\beta$-VAEs across the achievable range of rate-distortion trade-offs. Further, we show that Echo noise can outperform state-of-the-art flow methods without the need to train complex distributional transformations

该研究论文介绍了一种新的噪声通道—— Echo noise，该噪声通道不需要局限于分布假设，具有复杂的编码机制和精确的率规则化，能够在可实现的失真率权衡范围内占据支配地位，并能够在无需训练附加分布转换的情况下优于基于流的方法。

通过回声噪声在自编码器中精确实现速率-失真