Sustainable artificial intelligence focuses on data, hardware, and algorithms to make machine learning models more environmentally responsible. In particular, machine learning models for speech representations are computationally expensive, generating environmental concerns because of their high energy consumption. Thus, we propose a sustainable self-supervised model to learn speech representation, combining optimizations in neural layers and training to reduce computing costs. The proposed model improves over a resource-efficient baseline, reducing both memory usage and computing cost estimations. It pretrains using a single GPU in less than a day. On top of that, it improves the error rate performance of the baseline in downstream task evaluations. When comparing it to large speech representation approaches, there is an order of magnitude reduction in memory usage, while computing cost reductions represent almost three orders of magnitude improvement.

可持续人工智能聚焦于通过对数据、硬件和算法的优化来使机器学习模型更加环保。我们提出了一个可持续的自监督模型，用于学习语音表征，结合神经层和训练的优化以降低计算成本，并通过将其与资源高效模型进行比较，在内存使用和计算成本上实现了数量级的改进。

可持续的自我监督学习用于语音表示