Transformer-based models show state-of-the-art performance even for
large-scale Traveling Salesman Problems (TSPs). However, they are based on
fully-connected attention models and suffer from large computational complexity
and GPU memory usage. We propose a lightweight CNN-Transformer model based on a
CNN embedding layer and partial self-attention. Our CNN-Transformer model is
able to better learn spatial features from input data using a CNN embedding
layer compared with the standard Transformer models. It also removes
considerable redundancy in fully connected attention models using the proposed
partial self-attention. Experiments show that the proposed model outperforms
other state-of-the-art Transformer-based models in terms of TSP solution
quality, GPU memory usage, and inference time. Our model consumes approximately
20% less GPU memory usage and has 45% faster inference time compared with other
state-of-the-art Transformer-based models. Our code is publicly available at
this https URL

本研究提出了一种基于 CNN 嵌入层和局部自注意力的轻量级 CNN-Transformer 模型，相对于标准 Transformer 模型，该模型能够更好地从输入数据中学习空间特征，同时通过局部自注意力减少全连接注意模型中的冗余性。实验结果表明，我们提出的模型在解决 TSP 问题的质量、GPU 内存使用和推理时间等方面优于其他最先进的 Transformer-based 模型。