In-context learning provides a new perspective for multi-task modeling for
vision and NLP. Under this setting, the model can perceive tasks from prompts
and accomplish them without any extra task-specific head predictions or model
fine-tuning. However, Skeleton sequence modeling via in-context learning
remains unexplored. Directly applying existing in-context models from other
areas onto skeleton sequences fails due to the inter-frame and cross-task pose
similarity that makes it outstandingly hard to perceive the task correctly from
a subtle context. To address this challenge, we propose Skeleton-in-Context
(SiC), an effective framework for in-context skeleton sequence modeling. Our
SiC is able to handle multiple skeleton-based tasks simultaneously after a
single training process and accomplish each task from context according to the
given prompt. It can further generalize to new, unseen tasks according to
customized prompts. To facilitate context perception, we additionally propose a
task-unified prompt, which adaptively learns tasks of different natures, such
as partial joint-level generation, sequence-level prediction, or 2D-to-3D
motion prediction. We conduct extensive experiments to evaluate the
effectiveness of our SiC on multiple tasks, including motion prediction, pose
estimation, joint completion, and future pose estimation. We also evaluate its
generalization capability on unseen tasks such as motion-in-between. These
experiments show that our model achieves state-of-the-art multi-task
performance and even outperforms single-task methods on certain tasks.

通过上下文学习来进行骨架序列建模是一种新的视觉与自然语言处理多任务建模方法，本文提出了一种名为 Skeleton-in-Context (SiC) 的有效框架用于骨架序列的上下文建模，实现了多种基于骨架的任务的同时执行，并且可以根据给定提示进一步推广到新的未见任务。