Sparse auto-encoders are useful for extracting low-dimensional representations from high-dimensional data. However, their performance degrades sharply when the input noise at test time differs from the noise employed during training. This limitation hinders the applicability of auto-encoders in real-world scenarios where the level of noise in the input is unpredictable. In this paper, we formalize single hidden layer sparse auto-encoders as a transform learning problem. Leveraging the transform modeling interpretation, we propose an optimization problem that leads to a predictive model invariant to the noise level at test time. In other words, the same pre-trained model is able to generalize to different noise levels. The proposed optimization algorithm, derived from the square root lasso, is translated into a new, computationally efficient auto-encoding architecture. After proving that our new method is invariant to the noise level, we evaluate our approach by training networks using the proposed architecture for denoising tasks. Our experimental results demonstrate that the trained models yield a significant improvement in stability against varying types of noise compared to commonly used architectures.

稀疏自编码器在处理高维数据中提取低维表示方面具有实用性。然而，当测试时输入噪声与训练过程中使用的噪声不同时，其性能会显著降低。本文将单隐藏层稀疏自编码器形式化为一种转换学习问题，并提出了一个优化问题，导致了预测模型在测试时对噪声水平具有不变性。换句话说，同样的预训练模型能够泛化到不同的噪声水平。通过基于平方根 Lasso 的提出的优化算法，将其转换成一个新的计算效率高的自编码器架构。通过证明我们的新方法对噪声水平具有不变性，我们通过在去噪任务中使用提出的架构训练网络来评估我们的方法。实验结果表明，与常用的架构相比，训练模型在稳定性上在各种不同类型的噪声情况下有显著改善。

通过自归一化ReLU实现的关键自动编码器