We introduce a pioneering unified library that leverages depth anything,
segment anything models to augment neural comprehension in language-vision
model zero-shot understanding. This library synergizes the capabilities of the
Depth Anything Model (DAM), Segment Anything Model (SAM), and GPT-4V, enhancing
multimodal tasks such as vision-question-answering (VQA) and composition
reasoning. Through the fusion of segmentation and depth analysis at the
symbolic instance level, our library provides nuanced inputs for language
models, significantly advancing image interpretation. Validated across a
spectrum of in-the-wild real-world images, our findings showcase progress in
vision-language models through neural-symbolic integration. This novel approach
melds visual and language analysis in an unprecedented manner. Overall, our
library opens new directions for future research aimed at decoding the
complexities of the real world through advanced multimodal technologies and our
code is available at
https://github.com/AnthonyHuo/SAM-DAM-for-Compositional-Reasoning.

该研究介绍了一种开创性的统一库，利用深度模型和分割模型来增强语言 - 视觉模型的零 - shot 理解能力，通过分割和深度分析在符号实例水平上的融合，提供给语言模型细致入微的图像输入，显著推动了图像解释的进展，该库在野外真实图像的多个领域得到验证，展示了通过神经 - 符号一体化整合视觉和语言分析的新途径，为未来研究开启了新方向。