Stereo matching has become a key technique for 3D environment perception in
intelligent vehicles. For a considerable time, convolutional neural networks
(CNNs) have remained the mainstream choice for feature extraction in this
domain. Nonetheless, there is a growing consensus that the existing paradigm
should evolve towards vision foundation models (VFM), particularly those
developed based on vision Transformers (ViTs) and pre-trained through
self-supervision on extensive, unlabeled datasets. While VFMs are adept at
extracting informative, general-purpose visual features, specifically for dense
prediction tasks, their performance often lacks in geometric vision tasks. This
study serves as the first exploration of a viable approach for adapting VFMs to
stereo matching. Our ViT adapter, referred to as ViTAS, is constructed upon
three types of modules: spatial differentiation, patch attention fusion, and
cross-attention. The first module initializes feature pyramids, while the
latter two aggregate stereo and multi-scale contextual information into
fine-grained features, respectively. ViTAStereo, which combines ViTAS with cost
volume-based stereo matching back-end processes, achieves the top rank on the
KITTI Stereo 2012 dataset and outperforms the second-best network StereoBase by
approximately 7.9% in terms of the percentage of error pixels, with a tolerance
of 3 pixels. Additional experiments across diverse scenarios further
demonstrate its superior generalizability compared to all other
state-of-the-art approaches. We believe this new paradigm will pave the way for
the next generation of stereo matching networks.

该研究是首次探索一种可行的方法，将 VFMs 适应于立体匹配，并通过将 ViTAS 与基于代价体积的立体匹配后端处理相结合，在 KITTI Stereo 2012 数据集上取得了最高排名，并以误差像素比例方面优于次优网络 StereoBase 约 7.9％，3 个像素误差容差。在各种情境的附加实验进一步证明了其与所有其他最先进方法相比的卓越通用性。我们相信这种新范式将为下一代立体匹配网络铺平道路。