The latest advancements in unsupervised learning of sentence embeddings
predominantly involve employing contrastive learning-based (CL-based)
fine-tuning over pre-trained language models. In this study, we analyze the
latest sentence embedding methods by adopting representation rank as the
primary tool of analysis. We first define Phase 1 and Phase 2 of fine-tuning
based on when representation rank peaks. Utilizing these phases, we conduct a
thorough analysis and obtain essential findings across key aspects, including
alignment and uniformity, linguistic abilities, and correlation between
performance and rank. For instance, we find that the dynamics of the key
aspects can undergo significant changes as fine-tuning transitions from Phase 1
to Phase 2. Based on these findings, we experiment with a rank reduction (RR)
strategy that facilitates rapid and stable fine-tuning of the latest CL-based
methods. Through empirical investigations, we showcase the efficacy of RR in
enhancing the performance and stability of five state-of-the-art sentence
embedding methods.

本研究通过使用对比学习的 fine-tuning 方法并以表示排名作为主要分析工具，分析了最新的句子嵌入方法。通过定义 fine-tuning 的两个阶段，并利用这些阶段进行全面分析，我们得出了关于对齐和一致性、语言能力以及性能和排名之间相关性的重要发现。基于这些发现，我们提出了一种排名降低（RR）策略，以促进最新的对比学习方法的快速和稳定的 fine-tuning，并通过实证研究展示了 RR 在提高五种最先进句子嵌入方法的性能和稳定性方面的有效性。

揭示句子嵌入中微调的关键要素：表示排名分析

Unveiling Key Aspects of Fine-Tuning in Sentence Embeddings: A  Representation Rank Analysis

Representation rank is an important concept for understanding the role of
Neural Networks (NNs) in Deep Reinforcement learning (DRL), which measures the
expressive capacity of value networks. Existing studies focus on unboundedly
maximizing this rank; nevertheless, that approach would introduce overly
complex models in the learning, thus undermining performance. Hence,
fine-tuning representation rank presents a challenging and crucial optimization
problem. To address this issue, we find a guiding principle for adaptive
control of the representation rank. We employ the Bellman equation as a
theoretical foundation and derive an upper bound on the cosine similarity of
consecutive state-action pairs representations of value networks. We then
leverage this upper bound to propose a novel regularizer, namely BEllman
Equation-based automatic rank Regularizer (BEER). This regularizer adaptively
regularizes the representation rank, thus improving the DRL agent's
performance. We first validate the effectiveness of automatic control of rank
on illustrative experiments. Then, we scale up BEER to complex continuous
control tasks by combining it with the deterministic policy gradient method.
Among 12 challenging DeepMind control tasks, BEER outperforms the baselines by
a large margin. Besides, BEER demonstrates significant advantages in Q-value
approximation. Our code is available at
this https URL

通过引入新的正则化器 —— 基于 Bellman 方程的自动等级正则化器（BEER），我们可以自适应地调节表示等级，从而提高深度强化学习代理的性能。实验证明，BEER 在 12 项挑战性控制任务中表现出色，并且在 Q 值近似中也具有显著优势。