Parameter-efficient fine-tuning (PEFT) methods can adapt large language
models to downstream tasks by training a small amount of newly added
parameters. In multi-task settings, PEFT adapters typically train on each task
independently, inhibiting transfer across tasks, or on the concatenation of all
tasks, which can lead to negative interference. To address this, Polytropon
(Ponti et al.) jointly learns an inventory of PEFT adapters and a routing
function to share variable-size sets of adapters across tasks. Subsequently,
adapters can be re-combined and fine-tuned on novel tasks even with limited
data. In this paper, we investigate to what extent the ability to control which
adapters are active for each task leads to sample-efficient generalization.
Thus, we propose less expressive variants where we perform weighted averaging
of the adapters before few-shot adaptation (Poly-mu) instead of learning a
routing function. Moreover, we introduce more expressive variants where
finer-grained task-adapter allocation is learned through a multi-head routing
function (Poly-S). We test these variants on three separate benchmarks for
multi-task learning. We find that Poly-S achieves gains on all three (up to 5.3
points on average) over strong baselines, while incurring a negligible
additional cost in parameter count. In particular, we find that instruction
tuning, where models are fully fine-tuned on natural language instructions for
each task, is inferior to modular methods such as Polytropon and our proposed
variants.

本文提出了 Poly-mu 和 Poly-S 两种新的方法，对比了它们与 Polytropon 方法在多个任务学习中的表现，并发现 Poly-S 方法可以在保持参数少量增加的同时，在三个测试数据集上获得了高达 5.3 个点的平均收益。

数据高效调优的多头适配器路由

Multi-Head Adapter Routing for Data-Efficient Fine-Tuning

Simple weighted averaging of word vectors often yields effective
representations for sentences which outperform sophisticated seq2seq neural
models in many tasks. While it is desirable to use the same method to represent
documents as well, unfortunately, the effectiveness is lost when representing
long documents involving multiple sentences. One of the key reasons is that a
longer document is likely to contain words from many different topics; hence,
creating a single vector while ignoring all the topical structure is unlikely
to yield an effective document representation. This problem is less acute in
single sentences and other short text fragments where the presence of a single
topic is most likely. To alleviate this problem, we present P-SIF, a
partitioned word averaging model to represent long documents. P-SIF retains the
simplicity of simple weighted word averaging while taking a document's topical
structure into account. In particular, P-SIF learns topic-specific vectors from
a document and finally concatenates them all to represent the overall document.
We provide theoretical justifications on the correctness of P-SIF. Through a
comprehensive set of experiments, we demonstrate P-SIF's effectiveness compared
to simple weighted averaging and many other baselines.

P-SIF 是一种基于分区词向量加权平均的长文档表示方法，相较于简单的加权平均和其他基准方法，在理论上和实际效果上都更为有效。

P-SIF: 使用划分平均法进行文档嵌入

P-SIF: Document Embeddings Using Partition Averaging

Joint image filters are used to transfer structural details from a guidance
picture used as a prior to a target image, in tasks such as enhancing spatial
resolution and suppressing noise. Previous methods based on convolutional
neural networks (CNNs) combine nonlinear activations of spatially-invariant
kernels to estimate structural details and regress the filtering result. In
this paper, we instead learn explicitly sparse and spatially-variant kernels.
We propose a CNN architecture and its efficient implementation, called the
deformable kernel network (DKN), that outputs sets of neighbors and the
corresponding weights adaptively for each pixel. The filtering result is then
computed as a weighted average. We also propose a fast version of DKN that runs
about seventeen times faster for an image of size 640 x 480. We demonstrate the
effectiveness and flexibility of our models on the tasks of depth map
upsampling, saliency map upsampling, cross-modality image restoration, texture
removal, and semantic segmentation. In particular, we show that the weighted
averaging process with sparsely sampled 3 x 3 kernels outperforms the state of
the art by a significant margin in all cases.

该论文提出了 DKN 卷积神经网络架构及其高效实现，使用稀疏且空间可变的滤波核进行图像增强和降噪，并在深度图上采样，显著优于现有技术。