Neural speech codec has recently gained widespread attention in generative
speech modeling domains, like voice conversion, text-to-speech synthesis, etc.
However, ensuring high-fidelity audio reconstruction of speech codecs under
high compression rates remains an open and challenging issue. In this paper, we
propose PromptCodec, a novel end-to-end neural speech codec model using
disentangled representation learning based feature-aware prompt encoders. By
incorporating additional feature representations from prompt encoders,
PromptCodec can distribute the speech information requiring processing and
enhance its capabilities. Moreover, a simple yet effective adaptive feature
weighted fusion approach is introduced to integrate features of different
encoders. Meanwhile, we propose a novel disentangled representation learning
strategy based on cosine distance to optimize PromptCodec's encoders to ensure
their efficiency, thereby further improving the performance of PromptCodec.
Experiments on LibriTTS demonstrate that our proposed PromptCodec consistently
outperforms state-of-the-art neural speech codec models under all different
bitrate conditions while achieving impressive performance with low bitrates.

利用解耦表示学习为基准的特征感知提示编码器，我们提出了 PromptCodec，这是一种创新的端到端神经语音编解码模型，用于高压缩比下确保音频重建的高保真度。PromptCodec 通过整合提示编码器的额外特征表示，可以分配需要处理的语音信息并提高其能力。此外，我们引入了一种简单但有效的自适应特征加权融合方法，用于整合不同编码器的特征，并基于余弦距离提出了一种新颖的解耦表示学习策略以优化 PromptCodec 的编码器，从而进一步提高其性能。在 LibriTTS 上的实验证明，我们提出的 PromptCodec 在不同比特率条件下始终优于最先进的神经语音编解码模型，并在低比特率下取得了令人印象深刻的性能。