In this paper, we present a novel methodology we call MDS-ViTNet (Multi
Decoder Saliency by Vision Transformer Network) for enhancing visual saliency
prediction or eye-tracking. This approach holds significant potential for
diverse fields, including marketing, medicine, robotics, and retail. We propose
a network architecture that leverages the Vision Transformer, moving beyond the
conventional ImageNet backbone. The framework adopts an encoder-decoder
structure, with the encoder utilizing a Swin transformer to efficiently embed
most important features. This process involves a Transfer Learning method,
wherein layers from the Vision Transformer are converted by the Encoder
Transformer and seamlessly integrated into a CNN Decoder. This methodology
ensures minimal information loss from the original input image. The decoder
employs a multi-decoding technique, utilizing dual decoders to generate two
distinct attention maps. These maps are subsequently combined into a singular
output via an additional CNN model. Our trained model MDS-ViTNet achieves
state-of-the-art results across several benchmarks. Committed to fostering
further collaboration, we intend to make our code, models, and datasets
accessible to the public.

本文介绍了一种名为 MDS-ViTNet（多解码视觉变换网络）的新方法，用于增强视觉显著性预测和眼动跟踪。该方法在多个领域具有重要潜力，包括市场营销、医学、机器人技术和零售业。我们提出了一种利用 Vision Transformer 的网络架构，超越传统的 ImageNet 骨干。该框架采用编码器 - 解码器结构，其中编码器利用 Swin 变换器有效地嵌入最重要的特征。通过转换器将 Vision Transformer 的层转换并无缝集成到 CNN 解码器中，从而实现了该过程。编码器 - 解码器方法能够最小化原始输入图像的信息损失。解码器采用多解码技术，利用双解码器生成两个不同的关注度图。通过附加的 CNN 模型将这些图合并为单一输出。我们训练的模型 MDS-ViTNet 在多个基准测试中实现了最新的结果。为了促进进一步的合作，我们计划将我们的代码、模型和数据集开放给公众使用。