Language-queried audio source separation (LASS) is a new paradigm for
computational auditory scene analysis (CASA). LASS aims to separate a target
sound from an audio mixture given a natural language query, which provides a
natural and scalable interface for digital audio applications. Recent works on
LASS, despite attaining promising separation performance on specific sources
(e.g., musical instruments, limited classes of audio events), are unable to
separate audio concepts in the open domain. In this work, we introduce
AudioSep, a foundation model for open-domain audio source separation with
natural language queries. We train AudioSep on large-scale multimodal datasets
and extensively evaluate its capabilities on numerous tasks including audio
event separation, musical instrument separation, and speech enhancement.
AudioSep demonstrates strong separation performance and impressive zero-shot
generalization ability using audio captions or text labels as queries,
substantially outperforming previous audio-queried and language-queried sound
separation models. For reproducibility of this work, we will release the source
code, evaluation benchmark and pre-trained model at:
this https URL

通过自然语言查询实现了语言查询音频源分离（LASS），并为开放领域的音频分隔提供了 AudioSep 基础模型，展示了强大的分隔性能和令人印象深刻的零 - shot 推广能力。

描述的任何事物分开

Separate Anything You Describe

Autoregressive models have achieved impressive results over a wide range of
domains in terms of generation quality and downstream task performance. In the
continuous domain, a key factor behind this success is the usage of quantized
latent spaces (e.g., obtained via VQ-VAE autoencoders), which allow for
dimensionality reduction and faster inference times. However, using existing
pre-trained models to perform new non-trivial tasks is difficult since it
requires additional fine-tuning or extensive training to elicit prompting. This
paper introduces LASS as a way to perform vector-quantized Latent
Autoregressive Source Separation (i.e., de-mixing an input signal into its
constituent sources) without requiring additional gradient-based optimization
or modifications of existing models. Our separation method relies on the
Bayesian formulation in which the autoregressive models are the priors, and a
discrete (non-parametric) likelihood function is constructed by performing
frequency counts over latent sums of addend tokens. We test our method on
images and audio with several sampling strategies (e.g., ancestral, beam
search) showing competitive results with existing approaches in terms of
separation quality while offering at the same time significant speedups in
terms of inference time and scalability to higher dimensional data.

本论文介绍了一种名为 LASS 的矢量量化潜在自回归源分离方法，通过贝叶斯公式和构建离散概率密度函数来实现无需额外微调和修改现有模型即可将输入信号分离成其成分源，该方法在图像和音频分离方面表现出具有竞争力的结果。