In this paper, we introduce the task of language-queried audio source
separation (LASS), which aims to separate a target source from an audio mixture
based on a natural language query of the target source (e.g., "a man tells a
joke followed by people laughing"). A unique challenge in LASS is associated
with the complexity of natural language description and its relation with the
audio sources. To address this issue, we proposed LASS-Net, an end-to-end
neural network that is learned to jointly process acoustic and linguistic
information, and separate the target source that is consistent with the
language query from an audio mixture. We evaluate the performance of our
proposed system with a dataset created from the AudioCaps dataset. Experimental
results show that LASS-Net achieves considerable improvements over baseline
methods. Furthermore, we observe that LASS-Net achieves promising
generalization results when using diverse human-annotated descriptions as
queries, indicating its potential use in real-world scenarios. The separated
audio samples and source code are available at
this https URL.

该研究介绍了一种语言查询音频源分离（LASS）任务，该任务通过基于目标源的自然语言查询（例如，“一个男人讲笑话，接着人们笑了”）从音频混合物中分离出目标源。为了解决这个问题，作者提出了一个名为 LASS-Net 的端到端神经网络，它能够共同处理声学和语言信息，并从音频混合物中分离出与语言查询一致的目标源。实验结果表明，LASS-Net 相较于基线方法有明显的提升，并展示了良好的泛化性能，表明其在实际场景中的潜力。