The paper presents a novel two-stream network architecture for enhancing
scene understanding in computer vision. This architecture utilizes a graph
feature stream and an image feature stream, aiming to merge the strengths of
both modalities for improved performance in image classification and scene
graph generation tasks. The graph feature stream network comprises a
segmentation structure, scene graph generation, and a graph representation
module. The segmentation structure employs the UPSNet architecture with a
backbone that can be a residual network, Vit, or Swin Transformer. The scene
graph generation component focuses on extracting object labels and neighborhood
relationships from the semantic map to create a scene graph. Graph
Convolutional Networks (GCN), GraphSAGE, and Graph Attention Networks (GAT) are
employed for graph representation, with an emphasis on capturing node features
and their interconnections. The image feature stream network, on the other
hand, focuses on image classification through the use of Vision Transformer and
Swin Transformer models. The two streams are fused using various data fusion
methods. This fusion is designed to leverage the complementary strengths of
graph-based and image-based features.Experiments conducted on the ADE20K
dataset demonstrate the effectiveness of the proposed two-stream network in
improving image classification accuracy compared to conventional methods. This
research provides a significant contribution to the field of computer vision,
particularly in the areas of scene understanding and image classification, by
effectively combining graph-based and image-based approaches.

本论文提出了一种新颖的双流网络架构，通过结合图像特征流和图特征流，旨在提高图像分类和场景图生成任务的性能。利用多种数据融合方法将这两个流融合起来，以充分发挥基于图和基于图像的特征的互补优势。实验证明，该双流网络在提高图像分类准确性方面优于传统方法，对计算机视觉领域，尤其是场景理解和图像分类领域具有重要贡献。