There is a growing interest in cost-effective self-supervised fine-tuning
(SSFT) of self-supervised learning (SSL)-based speech models to obtain
task-specific representations. These task-specific representations are used for
robust performance on various downstream tasks by fine-tuning on the labelled
data. This work presents a cost-effective SSFT method named Self-supervised
Correspondence (SCORE) fine-tuning to adapt the SSL speech representations for
content-related tasks. The proposed method uses a correspondence training
strategy, aiming to learn similar representations from perturbed speech and
original speech. Commonly used data augmentation techniques for content-related
tasks (ASR) are applied to obtain perturbed speech. SCORE fine-tuned HuBERT
outperforms the vanilla HuBERT on SUPERB benchmark with only a few hours of
fine-tuning (< 5 hrs) on a single GPU for automatic speech recognition, phoneme
recognition, and query-by-example tasks, with relative improvements of 1.09%,
3.58%, and 12.65%, respectively. SCORE provides competitive results with the
recently proposed SSFT method SPIN, using only 1/3 of the processed speech
compared to SPIN.

提出了一种称为 SCORE 微调的自我监督一致性微调方法，用于自适应自我监督学习语音表示，以适应与内容相关的任务。与 SPIN 方法相比，SCORE 仅使用 1/3 的处理语音，在超级基准（SUPERB benchmark）上表现出优越的结果。(Translated simplified Chinese summary)

SCORE：自监督对应微调以改善内容表达

SCORE: Self-supervised Correspondence Fine-tuning for Improved Content  Representations

Neural fields have become widely used in various fields, from shape
representation to neural rendering, and for solving partial differential
equations (PDEs). With the advent of hybrid neural field representations like
Instant NGP that leverage small MLPs and explicit representations, these models
train quickly and can fit large scenes. Yet in many applications like rendering
and simulation, hybrid neural fields can cause noticeable and unreasonable
artifacts. This is because they do not yield accurate spatial derivatives
needed for these downstream applications. In this work, we propose two ways to
circumvent these challenges. Our first approach is a post hoc operator that
uses local polynomial-fitting to obtain more accurate derivatives from
pre-trained hybrid neural fields. Additionally, we also propose a
self-supervised fine-tuning approach that refines the neural field to yield
accurate derivatives directly while preserving the initial signal. We show the
application of our method on rendering, collision simulation, and solving PDEs.
We observe that using our approach yields more accurate derivatives, reducing
artifacts and leading to more accurate simulations in downstream applications.

我们提出了两种方法来解决混合神经场在渲染、碰撞模拟和解决偏微分方程等下游应用中可能引起的不合理现象和伪影问题：首先，我们提出了一种后处理算子，利用局部多项式拟合从预训练的混合神经场中获得更准确的导数；其次，我们提出了一种自监督微调方法，通过对神经场进行微调来直接获得准确的导数，同时保持初始信号的不变性。使用我们的方法可以获得更准确的导数，减少伪影，并在下游应用中实现更准确的模拟。