Self-supervised monocular depth estimation is of significant importance with
applications spanning across autonomous driving and robotics. However, the
reliance on self-supervision introduces a strong static-scene assumption,
thereby posing challenges in achieving optimal performance in dynamic scenes,
which are prevalent in most real-world situations. To address these issues, we
propose PPEA-Depth, a Progressive Parameter-Efficient Adaptation approach to
transfer a pre-trained image model for self-supervised depth estimation. The
training comprises two sequential stages: an initial phase trained on a dataset
primarily composed of static scenes, succeeded by an expansion to more
intricate datasets involving dynamic scenes. To facilitate this process, we
design compact encoder and decoder adapters to enable parameter-efficient
tuning, allowing the network to adapt effectively. They not only uphold
generalized patterns from pre-trained image models but also retain knowledge
gained from the preceding phase into the subsequent one. Extensive experiments
demonstrate that PPEA-Depth achieves state-of-the-art performance on KITTI,
CityScapes and DDAD datasets.

自监督单目深度估计在自动驾驶和机器人领域具有重要意义。本文提出了一种渐进参数高效适应的方法，通过转移预训练的图像模型来实现自监督深度估计，解决了动态场景中的性能优化挑战，并通过在 KITTI、CityScapes 和 DDAD 数据集上进行广泛实验，取得了最先进的效果。

PPEA-Depth: 面向自监督单目深度估计的渐进式参数高效调整

PPEA-Depth: Progressive Parameter-Efficient Adaptation for  Self-Supervised Monocular Depth Estimation

Fine-tuning large language models is becoming ever more impractical due to
their rapidly-growing scale. This motivates the use of parameter-efficient
adaptation methods such as prompt tuning (PT), which adds a small number of
tunable embeddings to an otherwise frozen model, and in-context learning (ICL),
in which demonstrations of the task are provided to the model in natural
language without any additional training. Recently, Singhal et al. (2022)
propose ``instruction prompt tuning'' (IPT), which combines PT with ICL by
concatenating a natural language demonstration with learned prompt embeddings.
While all of these methods have proven effective on different tasks, how they
interact with each other remains unexplored. In this paper, we empirically
study when and how in-context examples improve prompt tuning by measuring the
effectiveness of ICL, PT, and IPT on five text generation tasks with multiple
base language models. We observe that (1) IPT does \emph{not} always outperform
PT, and in fact requires the in-context demonstration to be semantically
similar to the test input to yield improvements; (2) PT is unstable and
exhibits high variance, but combining PT and ICL (into IPT) consistently
reduces variance across all five tasks; and (3) prompts learned for a specific
source task via PT exhibit positive transfer when paired with in-context
examples of a different target task. Our results offer actionable insights on
choosing a suitable parameter-efficient adaptation method for a given task.

该研究通过实验测量了几种不同的文本生成任务原始模型的表现，对比分析了几种参数有效的适应方法（如提示调整、上下文学习和指导性提示调整），并通过 IPT 探究了这些方法之间的交互作用和优缺点。

上下文学习如何帮助提示调整？

How Does In-Context Learning Help Prompt Tuning?

In this paper, we explore the challenging problem of performing a generative
task in a target language when labeled data is only available in English, using
summarization as a case study. We assume a strict setting with no access to
parallel data or machine translation and find that common transfer learning
approaches struggle in this setting, as a generative multilingual model
fine-tuned purely on English catastrophically forgets how to generate
non-English. Given the recent rise of parameter-efficient adaptation
techniques, we conduct the first investigation into how one such method, prompt
tuning (Lester et al., 2021), can overcome catastrophic forgetting to enable
zero-shot cross-lingual generation. Our experiments show that
parameter-efficient prompt tuning provides gains over standard fine-tuning when
transferring between less-related languages, e.g., from English to Thai.
However, a significant gap still remains between these methods and
fully-supervised baselines. To improve cross-lingual transfer further, we
explore several approaches, including: (1) mixing in unlabeled multilingual
data, and (2) explicitly factoring prompts into recombinable language and task
components. Our approaches can provide further quality gains, suggesting that
robust zero-shot cross-lingual generation is within reach.

本文研究如何在只有英文标记数据的情况下，在目标语言中执行生成任务，以 summarization 作为案例研究。研究发现了通用的迁移学习方法在这种情况下存在困难，并且展示通过 prompt tuning 等方法可以实现跨语言生成。