In recent years, research on music transcription has focused mainly on
architecture design and instrument-specific data acquisition. With the lack of
availability of diverse datasets, progress is often limited to solo-instrument
tasks such as piano transcription. Several works have explored multi-instrument
transcription as a means to bolster the performance of models on low-resource
tasks, but these methods face the same data availability issues. We propose
Timbre-Trap, a novel framework which unifies music transcription and audio
reconstruction by exploiting the strong separability between pitch and timbre.
We train a single U-Net to simultaneously estimate pitch salience and
reconstruct complex spectral coefficients, selecting between either output
during the decoding stage via a simple switch mechanism. In this way, the model
learns to produce coefficients corresponding to timbre-less audio, which can be
interpreted as pitch salience. We demonstrate that the framework leads to
performance comparable to state-of-the-art instrument-agnostic transcription
methods, while only requiring a small amount of annotated data.

Timbre-Trap 是一个新型框架，通过利用音高和音色之间的强分离性，将音乐转录和音频重建相统一。我们通过训练一个 U-Net 模型，同时估计音高显著性和重建复杂谱系数，通过简单的切换机制在解码阶段选择其中之一的输出。我们证明了该框架的性能可与最先进的无特定乐器转录方法相媲美，而只需要少量的带注释数据。