Large Vision-Language Models (LVLMs) have demonstrated outstanding
performance in various general multimodal applications such as image
recognition and visual reasoning, and have also shown promising potential in
specialized domains. However, the application potential of LVLMs in the
insurance domain-characterized by rich application scenarios and abundant
multimodal data-has not been effectively explored. There is no systematic
review of multimodal tasks in the insurance domain, nor a benchmark
specifically designed to evaluate the capabilities of LVLMs in insurance. This
gap hinders the development of LVLMs within the insurance domain. In this
paper, we systematically review and distill multimodal tasks for four
representative types of insurance: auto insurance, property insurance, health
insurance, and agricultural insurance. We propose INS-MMBench, the first
comprehensive LVLMs benchmark tailored for the insurance domain. INS-MMBench
comprises a total of 2.2K thoroughly designed multiple-choice questions,
covering 12 meta-tasks and 22 fundamental tasks. Furthermore, we evaluate
multiple representative LVLMs, including closed-source models such as GPT-4o
and open-source models like BLIP-2. This evaluation not only validates the
effectiveness of our benchmark but also provides an in-depth performance
analysis of current LVLMs on various multimodal tasks in the insurance domain.
We hope that INS-MMBench will facilitate the further application of LVLMs in
the insurance domain and inspire interdisciplinary development. Our dataset and
evaluation code are available at this https URL

本研究系统评估和总结了 LVLMs 在保险领域中的多模态任务，并提出了 INS-MMBench 作为第一个专门为保险领域量身定制的全面 benchmark，该评估不仅验证了我们的 benchmark 的有效性，还对当前 LVLMs 在保险领域的各种多模态任务中进行了深入的性能分析。

INS-MMBench: 保险领域 LVLMs 性能综合评估的全面基准

INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance  in Insurance

We apply a general deep learning framework to address the non-factoid
question answering task. Our approach does not rely on any linguistic tools and
can be applied to different languages or domains. Various architectures are
presented and compared. We create and release a QA corpus and setup a new QA
task in the insurance domain. Experimental results demonstrate superior
performance compared to the baseline methods and various technologies give
further improvements. For this highly challenging task, the top-1 accuracy can
reach up to 65.3% on a test set, which indicates a great potential for
practical use.

应用深度学习框架解决非事实类问答任务，在保险领域创建和发布了问答语料库，实验结果表明性能优于基线方法和其他技术，对于这个高度具有挑战性的任务，测试集的最高准确率可达到 65.3％，具有广泛的实用潜力。