The success of Large Language Models (LLMs) has led to a parallel rise in the
development of Large Multimodal Models (LMMs), such as Gemini-pro, which have
begun to transform a variety of applications. These sophisticated multimodal
models are designed to interpret and analyze complex data, integrating both
textual and visual information on a scale previously unattainable, opening new
avenues for a range of applications. This paper investigates the applicability
and effectiveness of prompt-engineered Gemini-pro LMMs versus fine-tuned Vision
Transformer (ViT) models in addressing critical security challenges. We focus
on two distinct tasks: a visually evident task of detecting simple triggers,
such as small squares in images, indicative of potential backdoors, and a
non-visually evident task of malware classification through visual
representations. Our results highlight a significant divergence in performance,
with Gemini-pro falling short in accuracy and reliability when compared to
fine-tuned ViT models. The ViT models, on the other hand, demonstrate
exceptional accuracy, achieving near-perfect performance on both tasks. This
study not only showcases the strengths and limitations of prompt-engineered
LMMs in cybersecurity applications but also emphasizes the unmatched efficacy
of fine-tuned ViT models for precise and dependable tasks.

研究了用于解决关键安全挑战的提示引擎 Gemini-pro 多模态模型和精调 ViT 模型的适用性和效果，发现 Gemini-pro 在准确性和可靠性方面存在明显差异，而精调的 ViT 模型在两个任务上表现出色，达到近乎完美的性能。