Recently, prompt-based methods have emerged as a new alternative
`parameter-efficient fine-tuning' paradigm, which only fine-tunes a small
number of additional parameters while keeping the original model frozen.
However, despite achieving notable results, existing prompt methods mainly
focus on `what to add', while overlooking the equally important aspect of
`where to add', typically relying on the manually crafted placement. To this
end, we propose a region-based Adaptive Visual Prompt, named AdaViPro, which
integrates the `where to add' optimization of the prompt into the learning
process. Specifically, we reconceptualize the `where to add' optimization as a
problem of regional decision-making. During inference, AdaViPro generates a
regionalized mask map for the whole image, which is composed of 0 and 1, to
designate whether to apply or discard the prompt in each specific area.
Therefore, we employ Gumbel-Softmax sampling to enable AdaViPro's end-to-end
learning through standard back-propagation. Extensive experiments demonstrate
that our AdaViPro yields new efficiency and accuracy trade-offs for adapting
pre-trained models.

我们提出了一个名为 AdaViPro 的区域自适应视觉提示，它将提示的 “添加到哪里” 优化集成到学习过程中，通过基于 Gumbel-Softmax 采样实现端到端学习，通过大量实验证明，AdaViPro 在适应预训练模型方面产生了新的效率和准确性权衡。

AdaViPro: 基于区域的大规模模型的自适应视觉提示

AdaViPro: Region-based Adaptive Visual Prompt for Large-Scale Models  Adapting

Diverse human motion prediction aims at predicting multiple possible future
pose sequences from a sequence of observed poses. Previous approaches usually
employ deep generative networks to model the conditional distribution of data,
and then randomly sample outcomes from the distribution. While different
results can be obtained, they are usually the most likely ones which are not
diverse enough. Recent work explicitly learns multiple modes of the conditional
distribution via a deterministic network, which however can only cover a fixed
number of modes within a limited range. In this paper, we propose a novel
sampling strategy for sampling very diverse results from an imbalanced
multimodal distribution learned by a deep generative model. Our method works by
generating an auxiliary space and smartly making randomly sampling from the
auxiliary space equivalent to the diverse sampling from the target
distribution. We propose a simple yet effective network architecture that
implements this novel sampling strategy, which incorporates a Gumbel-Softmax
coefficient matrix sampling method and an aggressive diversity promoting hinge
loss function. Extensive experiments demonstrate that our method significantly
improves both the diversity and accuracy of the samplings compared with
previous state-of-the-art sampling approaches. Code and pre-trained models are
available at this https URL

该文章提出了一种基于 Gumbel-Softmax 采样和 hinge 损失函数，通过在辅助空间内进行随机采样来实现从深度生成模型中多样化采样的新策略，并通过大量实验证明其在人体运动预测方面的效果显著优于现有策略

通过在辅助空间中使用 Gumbel-Softmax 抽样进行多样化的人体动作预测

Diverse Human Motion Prediction via Gumbel-Softmax Sampling from an  Auxiliary Space

In Grammatical Error Correction (GEC), sequence labeling models enjoy fast
inference compared to sequence-to-sequence models; however, inference in
sequence labeling GEC models is an iterative process, as sentences are passed
to the model for multiple rounds of correction, which exposes the model to
sentences with progressively fewer errors at each round. Traditional GEC models
learn from sentences with fixed error rates. Coupling this with the iterative
correction process causes a mismatch between training and inference that
affects final performance. In order to address this mismatch, we propose a
GAN-like sequence labeling model, which consists of a grammatical error
detector as a discriminator and a grammatical error labeler with Gumbel-Softmax
sampling as a generator. By sampling from real error distributions, our errors
are more genuine compared to traditional synthesized GEC errors, thus
alleviating the aforementioned mismatch and allowing for better training. Our
results on several evaluation benchmarks demonstrate that our proposed approach
is effective and improves the previous state-of-the-art baseline.

本研究提出了一种 GAN 类似的序列标注模型，采用 Gumbel-Softmax 采样和真实的错误分布来进行语法纠错，以解决传统模型训练与推理不匹配的问题，实验结果表明该方法有效，并改进了之前的最先进基准。