World models empower model-based agents to interactively explore, reason, and
plan within imagined environments for real-world decision-making. However, the
high demand for interactivity poses challenges in harnessing recent
advancements in video generative models for developing world models at scale.
This work introduces Interactive VideoGPT (iVideoGPT), a scalable
autoregressive transformer framework that integrates multimodal signals--visual
observations, actions, and rewards--into a sequence of tokens, facilitating an
interactive experience of agents via next-token prediction. iVideoGPT features
a novel compressive tokenization technique that efficiently discretizes
high-dimensional visual observations. Leveraging its scalable architecture, we
are able to pre-train iVideoGPT on millions of human and robotic manipulation
trajectories, establishing a versatile foundation that is adaptable to serve as
interactive world models for a wide range of downstream tasks. These include
action-conditioned video prediction, visual planning, and model-based
reinforcement learning, where iVideoGPT achieves competitive performance
compared with state-of-the-art methods. Our work advances the development of
interactive general world models, bridging the gap between generative video
models and practical model-based reinforcement learning applications.

通过引入 Interactive VideoGPT (iVideoGPT) 框架，可以在实现世界决策过程中，使模型交互地探索、推理和规划。 iVideoGPT 通过将多模态信号（视觉观察，行为和奖励）整合成一系列令牌，实现了通过下一个令牌预测的交互式体验，其中包括一种高效离散化高维视觉观察的新型压缩词汇化技术。借助其可扩展的架构，我们能够在数百万人类和机器人操作轨迹上进行预训练，建立起适用于各种下游任务的多功能基础，其中包括动作条件的视频预测，可视化规划和基于模型的强化学习，并且在与最先进方法相比，iVideoGPT 实现了竞争性的性能。我们的工作推进了交互式通用世界模型的发展，弥合了生成式视频模型和实际模型导向强化学习应用之间的差距。

iVideoGPT: 可扩展世界模型的交互式 VideoGPT

iVideoGPT: Interactive VideoGPTs are Scalable World Models

Human-AI interactivity is a critical aspect that reflects the usability of
multimodal large language models (MLLMs). However, existing end-to-end MLLMs
only allow users to interact with them through language instructions, leading
to the limitation of the interactive accuracy and efficiency. In this study, we
present precise referring instructions that utilize diverse reference
representations such as points and boxes as referring prompts to refer to the
special region. This enables MLLMs to focus on the region of interest and
achieve finer-grained interaction. Based on precise referring instruction, we
propose ChatSpot, a unified end-to-end multimodal large language model that
supports diverse forms of interactivity including mouse clicks, drag-and-drop,
and drawing boxes, which provides a more flexible and seamless interactive
experience. We also construct a multi-grained vision-language
instruction-following dataset based on existing datasets and GPT-4 generating.
Furthermore, we design a series of evaluation tasks to assess the effectiveness
of region recognition and interaction. Experimental results showcase ChatSpot's
promising performance.

基于精确的指代指令，我们提出了 ChatSpot，这是一个统一的端到端多模态大型语言模型，支持多种形式的交互，包括鼠标点击、拖放和绘制矩形框，从而提供更灵活、无缝的交互体验。实验证明 ChatSpot 具有良好的性能。

ChatSpot: 通过精确引导指导调整，通过引导引导引导链接

ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring  Instruction Tuning

Virtual Reality has become a significant element of education throughout the
years. To understand the quality and advantages of these techniques, it is
important to understand how they were developed and evaluated. Since COVID-19,
the education system has drastically changed a lot. It has shifted from being
in a classroom with a whiteboard and projectors to having your own room in
front of your laptop in a virtual meeting. In this respect, virtual reality in
the laboratory or Virtual Laboratory is the main focus of this research, which
is intended to comprehend the work done in quality education from a distance
using VR. As per the findings of the study, adopting virtual reality in
education can help students learn more effectively and also help them increase
perspective, enthusiasm, and knowledge of complex notions by offering them an
interactive experience in which they can engage and learn more effectively.
This highlights the importance of a significant expansion of VR use in
learning, the majority of which employ scientific comparison approaches to
compare students who use VR to those who use the traditional method for
learning.

使用 Virtual Reality 技术来提供交互式学习体验可以帮助学生更有效地学习知识，并增加他们的视角、热情和知识，因此大力扩展 VR 在学习中的应用非常重要。