The human ability to easily solve multimodal tasks in context (i.e., with
only a few demonstrations or simple instructions), is what current multimodal
systems have largely struggled to imitate. In this work, we demonstrate that
the task-agnostic in-context learning capabilities of large multimodal models
can be significantly enhanced by effective scaling-up. We introduce Emu2, a
generative multimodal model with 37 billion parameters, trained on large-scale
multimodal sequences with a unified autoregressive objective. Emu2 exhibits
strong multimodal in-context learning abilities, even emerging to solve tasks
that require on-the-fly reasoning, such as visual prompting and object-grounded
generation. The model sets a new record on multiple multimodal understanding
tasks in few-shot settings. When instruction-tuned to follow specific
instructions, Emu2 further achieves new state-of-the-art on challenging tasks
such as question answering benchmarks for large multimodal models and
open-ended subject-driven generation. These achievements demonstrate that Emu2
can serve as a base model and general-purpose interface for a wide range of
multimodal tasks. Code and models are publicly available to facilitate future
research.

我们展示了一个 37 亿参数的生成式多模态模型 Emu2，通过在大规模多模态序列上训练，使模型具备了强大的多模态上下文学习能力，甚至能够解决需要实时推理的任务，如视觉提示和目标驱动生成。该模型在少样本情况下刷新了多个多模态理解任务的记录，并在指令调整后在问题回答和主题驱动生成等挑战性任务上达到了最新的最高水平。这些成就表明 Emu2 可以作为一个基准模型和广泛应用于各种多模态任务的通用接口。