We study multitask online learning in a setting where agents can only
exchange information with their neighbors on an arbitrary communication
network. We introduce $\texttt{MT-CO}_2\texttt{OL}$, a decentralized algorithm
for this setting whose regret depends on the interplay between the task
similarities and the network structure. Our analysis shows that the regret of
$\texttt{MT-CO}_2\texttt{OL}$ is never worse (up to constants) than the bound
obtained when agents do not share information. On the other hand, our bounds
significantly improve when neighboring agents operate on similar tasks. In
addition, we prove that our algorithm can be made differentially private with a
negligible impact on the regret when the losses are linear. Finally, we provide
experimental support for our theory.

我们在一个能够通过网络与邻居交换信息的设定中研究多任务在线学习。我们介绍了一种分散算法 $	exttt {MT-CO}_2	exttt {OL}$，其遗憾度取决于任务相似性和网络结构之间的相互作用。我们的分析表明，$	exttt {MT-CO}_2	exttt {OL}$ 的遗憾度（在常数范围内）永远不会比没有信息共享的情况更糟。另一方面，在邻近代理在相似任务上操作时，我们的边界显著改进。此外，我们证明了当损失是线性的时候，我们的算法可以以微不足道的遗憾影响实现差分隐私。最后，我们提供对我们理论的实验支持。

多任务在线学习：聆听邻域嗡嗡声

Multitask Online Learning: Listen to the Neighborhood Buzz

Foundation models have achieved great advances in multi-task learning with a
unified interface of unimodal and multimodal tasks. However, the potential of
such multi-task learners has not been exploited during transfer learning. In
this work, we present a universal parameter-efficient transfer learning method,
termed Predict-Interpolate Tuning ($\pi$-Tuning), for vision, language, and
vision-language tasks. It aggregates the parameters of lightweight
task-specific experts learned from similar tasks to aid the target downstream
task. The task similarities are predicted in a unified modality-independent
space, yielding a scalable graph to demonstrate task relationships.
$\pi$-Tuning has several appealing benefits. First, it flexibly explores both
intra- and inter-modal transferability between similar tasks to improve the
accuracy and robustness of transfer learning, especially in data-scarce
scenarios. Second, it offers a systematical solution for transfer learning with
multi-task prediction-and-then-interpolation, compatible with diverse types of
parameter-efficient experts, such as prompt and adapter. Third, an extensive
study of task-level mutual benefits on 14 unimodal and 6 multimodal datasets
shows that $\pi$-Tuning surpasses fine-tuning and other parameter-efficient
transfer learning methods both in full-shot and low-shot regimes. The task
graph also enables an in-depth interpretable analysis of task transferability
across modalities.

本文提出了一种用于视觉，语言和视觉语言任务的通用参数高效的迁移学习方法 Predict-Interpolate Tuning（π-Tuning），它通过学习类似任务的轻量级任务专家的参数来帮助目标下游任务，并在一个统一的无模态空间中预测任务的相似性，从而产生可缩放图表以演示任务关系。研究表明，与微调和其他参数高效的迁移学习方法相比，π-Tuning 在全射和低射击区域都具有更好的性能。