Large language models with instruction-following abilities have
revolutionized the field of artificial intelligence. These models show
exceptional generalizability to tackle various real-world tasks through their
natural language interfaces. However, their performance heavily relies on
high-quality exemplar data, which is often difficult to obtain. This challenge
is further exacerbated when it comes to multimodal instruction following. We
introduce TextBind, an almost annotation-free framework for empowering larger
language models with the multi-turn interleaved multimodal
instruction-following capabilities. Our approach requires only image-caption
pairs and generates multi-turn multimodal instruction-response conversations
from a language model. We release our dataset, model, and demo to foster future
research in the area of multimodal instruction following.

介绍了 TextBind，这是一个几乎无需标注的框架，可以为较大的语言模型赋予多轮交错的多模态指令跟随能力，通过仅使用图像 - 标题对生成多轮多模态指令 - 回应对话，从而旨在推动多模态指令跟随领域的未来研究。