In a challenge-response study, we subjected Google Bard to 64 visual
challenges designed to probe multimodal Large Language Models (LLMs). The
challenges spanned diverse categories, including "Visual Situational
Reasoning," "Visual Text Reasoning," and "Next Scene Prediction," among others,
to discern Bard's competence in melding visual and linguistic analyses. Our
findings indicate that Bard tends to rely on making educated guesses about
visuals, especially when determining cues from images. Unlike other models like
GPT4, Bard does not appear to rely on optical character recognition libraries
like Tesseract but recognizes text in complex images like deep learning models
such as Google Lens and Visual API. Significantly Bard can solve CAPTCHAs
visually that ChatGPT fails to understand, recommending Tesseract solutions.
Moreover, while the Bard model proposes solutions based on visual input, it
cannot recreate or modify the original visual objects to support its
conclusions. Bard fails to redraw ASCII art that the text can describe or
capture a simple Tic Tac Toe grid it claims to analyze for the next moves. This
study provides experimental insights into the current capacities and areas for
improvement in multimodal LLMs.

通过一项挑战 - 回应研究，我们对 Google Bard 进行了 64 个视觉挑战，旨在探查多模式大型语言模型（LLMs）的能力。我们的发现表明，Bard 在确定图像中的线索时更倾向于根据直觉做出猜测，并且不依赖于 OCR 库而使用类似 Google Lens 和 Visual API 的深度学习模型来识别复杂图像中的文本。然而，Bard 无法重新绘制 ASCII 艺术或解析简单的井字棋网格，这项研究为多模式 LLMs 的当前能力和改进方向提供了实验性见解。