The development of effective machine learning methodologies for enhancing the
efficiency and accuracy of clinical systems is crucial. Despite significant
research efforts, managing a plethora of diversified clinical tasks and
adapting to emerging new tasks remain significant challenges. This paper
presents a novel paradigm that employs a pre-trained large language model as a
universal clinical multi-task decoder. This approach leverages the flexibility
and diversity of language expressions to handle task topic variations and
associated arguments. The introduction of a new task simply requires the
addition of a new instruction template. We validate this framework across
hundreds of tasks, demonstrating its robustness in facilitating multi-task
predictions, performing on par with traditional multi-task learning and
single-task learning approaches. Moreover, it shows exceptional adaptability to
new tasks, with impressive zero-shot performance in some instances and superior
data efficiency in few-shot scenarios. This novel approach offers a unified
solution to manage a wide array of new and emerging tasks in clinical
applications.

该论文介绍了一种利用预训练的大型语言模型作为通用临床多任务解码器的新范例，用于管理和适应多样化的临床任务及新任务的挑战，并验证了该框架在数百个任务上的鲁棒性，表现出与传统多任务学习和单任务学习方法相当的性能，对新任务具有出色的零样本表现和很强的数据效率，为临床应用中的各种新任务提供了统一的解决方案。

大型语言模型作为通用临床多任务解码器

Large Language Model as a Universal Clinical Multi-task Decoder

Recently, the rise of large-scale vision-language pretrained models like
CLIP, coupled with the technology of Parameter-Efficient FineTuning (PEFT), has
captured substantial attraction in video action recognition. Nevertheless,
prevailing approaches tend to prioritize strong supervised performance at the
expense of compromising the models' generalization capabilities during
transfer. In this paper, we introduce a novel Multimodal, Multi-task CLIP
adapting framework named \name to address these challenges, preserving both
high supervised performance and robust transferability. Firstly, to enhance the
individual modality architectures, we introduce multimodal adapters to both the
visual and text branches. Specifically, we design a novel visual TED-Adapter,
that performs global Temporal Enhancement and local temporal Difference
modeling to improve the temporal representation capabilities of the visual
encoder. Moreover, we adopt text encoder adapters to strengthen the learning of
semantic label information. Secondly, we design a multi-task decoder with a
rich set of supervisory signals to adeptly satisfy the need for strong
supervised performance and generalization within a multimodal framework.
Experimental results validate the efficacy of our approach, demonstrating
exceptional performance in supervised learning while maintaining strong
generalization in zero-shot scenarios.

该研究介绍了一种名为 \name 的新型多模态、多任务 CLIP 自适应框架，通过引入多模态适配器和多任务解码器，实现强大的监督学习性能和在零样本场景中的强大泛化能力。

M2-CLIP: 视频动作识别的多模态多任务自适应框架

M2-CLIP: A Multimodal, Multi-task Adapting Framework for Video Action  Recognition

Appearance of dressed humans undergoes a complex geometric transformation
induced not only by the static pose but also by its dynamics, i.e., there
exists a number of cloth geometric configurations given a pose depending on the
way it has moved. Such appearance modeling conditioned on motion has been
largely neglected in existing human rendering methods, resulting in rendering
of physically implausible motion. A key challenge of learning the dynamics of
the appearance lies in the requirement of a prohibitively large amount of
observations. In this paper, we present a compact motion representation by
enforcing equivariance -- a representation is expected to be transformed in the
way that the pose is transformed. We model an equivariant encoder that can
generate the generalizable representation from the spatial and temporal
derivatives of the 3D body surface. This learned representation is decoded by a
compositional multi-task decoder that renders high fidelity time-varying
appearance. Our experiments show that our method can generate a temporally
coherent video of dynamic humans for unseen body poses and novel views given a
single view video.

本研究提出一种新的具有运动条件的人类动态外观模型，使用等变编码器从三维身体表面的空间和时间导数生成可推广的表征，再通过组合式多任务解码器渲染具有高保真度的时间变化外观的视频，实验结果表明，本方法能生成视角全新的人类动态视频。