Conversational generative vision models (CGVMs) like Visual ChatGPT (Wu et
al., 2023) have recently emerged from the synthesis of computer vision and
natural language processing techniques. These models enable more natural and
interactive communication between humans and machines, because they can
understand verbal inputs from users and generate responses in natural language
along with visual outputs. To make informed decisions about the usage and
deployment of these models, it is important to analyze their performance
through a suitable evaluation framework on realistic datasets. In this paper,
we present ConvGenVisMo, a framework for the novel task of evaluating CGVMs.
ConvGenVisMo introduces a new benchmark evaluation dataset for this task, and
also provides a suite of existing and new automated evaluation metrics to
evaluate the outputs. All ConvGenVisMo assets, including the dataset and the
evaluation code, will be made available publicly on GitHub.

本文提出了 ConvGenVisMo 框架，旨在评估对话生成视觉模型（CGVMs）的性能，并介绍了一组自动化评估度量标准。该框架包含一个新的基准评估数据集，并提供公开可用的 ConvGenVisMo 资产，包括数据集和评估代码。

对话生成视觉模型的评估

ConvGenVisMo: Evaluation of Conversational Generative Vision Models

Recent advancements in Natural Language Processing (NLP), particularly in
Large Language Models (LLMs), associated with deep learning-based computer
vision techniques, have shown substantial potential for automating a variety of
tasks. One notable model is Visual ChatGPT, which combines ChatGPT's LLM
capabilities with visual computation to enable effective image analysis. The
model's ability to process images based on textual inputs can revolutionize
diverse fields. However, its application in the remote sensing domain remains
unexplored. This is the first paper to examine the potential of Visual ChatGPT,
a cutting-edge LLM founded on the GPT architecture, to tackle the aspects of
image processing related to the remote sensing domain. Among its current
capabilities, Visual ChatGPT can generate textual descriptions of images,
perform canny edge and straight line detection, and conduct image segmentation.
These offer valuable insights into image content and facilitate the
interpretation and extraction of information. By exploring the applicability of
these techniques within publicly available datasets of satellite images, we
demonstrate the current model's limitations in dealing with remote sensing
images, highlighting its challenges and future prospects. Although still in
early development, we believe that the combination of LLMs and visual models
holds a significant potential to transform remote sensing image processing,
creating accessible and practical application opportunities in the field.

本论文研究了一种基于 GPT 框架的新型深度学习模型 Visual ChatGPT 在遥感领域图像处理中的应用，成功展示了该模型生成图像文本描述、进行边缘检测和拟合直线等的能力，但也揭示了该模型在遥感图像处理中的局限性和挑战。研究者相信该模型的应用在未来有望将遥感图像处理转变为便利与实际应用机会并存的领域。