The rapidly evolving sector of Multi-modal Large Language Models (MLLMs) is
at the forefront of integrating linguistic and visual processing in artificial
intelligence. This paper presents an in-depth comparative study of two
pioneering models: Google's Gemini and OpenAI's GPT-4V(ision). Our study
involves a multi-faceted evaluation of both models across key dimensions such
as Vision-Language Capability, Interaction with Humans, Temporal Understanding,
and assessments in both Intelligence and Emotional Quotients. The core of our
analysis delves into the distinct visual comprehension abilities of each model.
We conducted a series of structured experiments to evaluate their performance
in various industrial application scenarios, offering a comprehensive
perspective on their practical utility. We not only involve direct performance
comparisons but also include adjustments in prompts and scenarios to ensure a
balanced and fair analysis. Our findings illuminate the unique strengths and
niches of both models. GPT-4V distinguishes itself with its precision and
succinctness in responses, while Gemini excels in providing detailed, expansive
answers accompanied by relevant imagery and links. These understandings not
only shed light on the comparative merits of Gemini and GPT-4V but also
underscore the evolving landscape of multimodal foundation models, paving the
way for future advancements in this area. After the comparison, we attempted to
achieve better results by combining the two models. Finally, We would like to
express our profound gratitude to the teams behind GPT-4V and Gemini for their
pioneering contributions to the field. Our acknowledgments are also extended to
the comprehensive qualitative analysis presented in 'Dawn' by Yang et al. This
work, with its extensive collection of image samples, prompts, and
GPT-4V-related results, provided a foundational basis for our analysis.

这篇论文对谷歌的 Gemini 和 OpenAI 的 GPT-4V (ision) 两个创新模型进行了深入的比较研究，重点关注它们在视觉理解能力、与人类交互、时间理解以及智能和情感商数等关键领域的表现，发现 GPT-4V 以精准简洁的回答著称，而 Gemini 在提供详细广泛回答和相关图像和链接方面表现出色。这些研究成果不仅揭示了 Gemini 和 GPT-4V 的比较优势，也突显了多模式基础模型领域的发展，为未来的进展铺平了道路。

Gemini 对 GPT-4V：基于定性案例的视觉 - 语言模型初步比较与结合

Gemini vs GPT-4V: A Preliminary Comparison and Combination of  Vision-Language Models Through Qualitative Cases

In the real world, some of the most complex settings for learned agents
involve interaction with humans, who often exhibit suboptimal, unpredictable
behavior due to sophisticated biases. Agents that interact with people in such
settings end up influencing the actions that these people take. Our goal in
this work is to enable agents to leverage that influence to improve the human's
performance in collaborative tasks, as the task unfolds. Unlike prior work, we
do not assume online training with people (which tends to be too expensive and
unsafe), nor access to a high fidelity simulator of the environment. Our idea
is that by taking a variety of previously observed human-human interaction data
and labeling it with the task reward, offline reinforcement learning (RL) can
learn to combine components of behavior, and uncover actions that lead to more
desirable human actions. First, we show that offline RL can learn strategies to
influence and improve human behavior, despite those strategies not appearing in
the dataset, by utilizing components of diverse, suboptimal interactions. In
addition, we demonstrate that offline RL can learn influence that adapts with
humans, thus achieving long-term coordination with them even when their
behavior changes. We evaluate our proposed method with real people in the
Overcooked collaborative benchmark domain, and demonstrate successful
improvement in human performance.

本篇论文提出了一种离线强化学习方法，通过利用多样化的人机交互行为，在不需要在线训练或高保真模拟器的情况下，学习一些对人类行为产生积极影响的策略，从而提高人类在合作任务中的表现。该方法成功在 Overcooked 协作基准域中提高了人类的表现。