In the rapidly evolving landscape of AI research and application, Multimodal
Large Language Models (MLLMs) have emerged as a transformative force, adept at
interpreting and integrating information from diverse modalities such as text,
images, and Graphical User Interfaces (GUIs). Despite these advancements, the
nuanced interaction and understanding of GUIs pose a significant challenge,
limiting the potential of existing models to enhance automation levels. To
bridge this gap, this paper presents V-Zen, an innovative Multimodal Large
Language Model (MLLM) meticulously crafted to revolutionise the domain of GUI
understanding and grounding. Equipped with dual-resolution image encoders,
V-Zen establishes new benchmarks in efficient grounding and next-action
prediction, thereby laying the groundwork for self-operating computer systems.
Complementing V-Zen is the GUIDE dataset, an extensive collection of real-world
GUI elements and task-based sequences, serving as a catalyst for specialised
fine-tuning. The successful integration of V-Zen and GUIDE marks the dawn of a
new era in multimodal AI research, opening the door to intelligent, autonomous
computing experiences. This paper extends an invitation to the research
community to join this exciting journey, shaping the future of GUI automation.
In the spirit of open science, our code, data, and model will be made publicly
available, paving the way for multimodal dialogue scenarios with intricate and
precise interactions.

V-Zen 是一款创新的多模态大语言模型，通过双分辨率图像编码器在 GUI 理解和下一步行动预测方面取得了突破性的成果，并与 GUIDE 数据集成功集成，标志着多模态人工智能研究的新时代的来临。