The AI community has been pursuing algorithms known as artificial general
intelligence (AGI) that apply to any kind of real-world problem. Recently, chat
systems powered by large language models (LLMs) emerge and rapidly become a
promising direction to achieve AGI in natural language processing (NLP), but
the path towards AGI in computer vision (CV) remains unclear. One may owe the
dilemma to the fact that visual signals are more complex than language signals,
yet we are interested in finding concrete reasons, as well as absorbing
experiences from GPT and LLMs to solve the problem. In this paper, we start
with a conceptual definition of AGI and briefly review how NLP solves a wide
range of tasks via a chat system. The analysis inspires us that unification is
the next important goal of CV. But, despite various efforts in this direction,
CV is still far from a system like GPT that naturally integrates all tasks. We
point out that the essential weakness of CV lies in lacking a paradigm to learn
from environments, yet NLP has accomplished the task in the text world. We then
imagine a pipeline that puts a CV algorithm (i.e., an agent) in world-scale,
interactable environments, pre-trains it to predict future frames with respect
to its action, and then fine-tunes it with instruction to accomplish various
tasks. We expect substantial research and engineering efforts to push the idea
forward and scale it up, for which we share our perspectives on future research
directions.

本文提出将计算机视觉算法放入世界范围内的互动环境进行预训练，完成各种任务的想法，以解决计算机视觉在实现人工通用智能方面的困难