Transformers have captured growing attention in computer vision, thanks to its large capacity and global processing capabilities. However, transformers are data hungry, and their ability to generalize is constrained compared to Convolutional Neural Networks (ConvNets), especially when trained with limited data due to the absence of the built-in spatial inductive biases present in ConvNets. In this paper, we strive to optimally combine the strengths of both convolution and transformers for image classification tasks. Towards this end, we present a novel lightweight hybrid network that pairs Convolution with Transformers via Representation Learning Fusion and Multi-Level Feature Cross-Attention named CTRL-F. Our network comprises a convolution branch and a novel transformer module named multi-level feature cross-attention (MFCA). The MFCA module operates on multi-level feature representations obtained at different convolution stages. It processes small patch tokens and large patch tokens extracted from these multi-level feature representations via two separate transformer branches, where both branches communicate and exchange knowledge through cross-attention mechanism. We fuse the local responses acquired from the convolution path with the global responses acquired from the MFCA module using novel representation fusion techniques dubbed adaptive knowledge fusion (AKF) and collaborative knowledge fusion (CKF). Experiments demonstrate that our CTRL-F variants achieve state-of-the-art performance, whether trained from scratch on large data or even with low-data regime. For Instance, CTRL-F achieves top-1 accuracy of 82.24% and 99.91% when trained from scratch on Oxford-102 Flowers and PlantVillage datasets respectively, surpassing state-of-the-art models which showcase the robustness of our model on image classification tasks. Code at: https://github.com/hosamsherif/CTRL-F

通过将卷积和transformer的优势相结合，我们提出了一种轻量级混合网络CTRL-F，结合了Representation Learning Fusion和Multi-Level Feature Cross-Attention两种技术，用于图像分类任务，实验表明我们的模型在大规模数据和低数据条件下取得了最先进的性能。

CTRL-F：通过多级特征交互和表示学习融合，在图像分类中将卷积与Transformer结合