Convolutional neural networks (CNNs) with large kernels, drawing inspiration
from the key operations of vision transformers (ViTs), have demonstrated
impressive performance in various vision-based applications. To address the
issue of computational efficiency degradation in existing designs for
supporting large-kernel convolutions, an FPGA-based inference accelerator is
proposed for the efficient deployment of CNNs with arbitrary kernel sizes.
Firstly, a Z-flow method is presented to optimize the computing data flow by
maximizing data reuse opportunity. Besides, the proposed design, incorporating
the kernel-segmentation (Kseg) scheme, enables extended support for
large-kernel convolutions, significantly reducing the storage requirements for
overlapped data. Moreover, based on the analysis of typical block structures in
emerging CNNs, vertical-fused (VF) and horizontal-fused (HF) methods are
developed to optimize CNN deployments from both computation and transmission
perspectives. The proposed hardware accelerator, evaluated on Intel Arria 10
FPGA, achieves up to 3.91 times better DSP efficiency than prior art on the
same network. Particularly, it demonstrates efficient support for large-kernel
CNNs, achieving throughputs of 169.68 GOPS and 244.55 GOPS for RepLKNet-31 and
PyConvResNet-50, respectively, both of which are implemented on hardware for
the first time.

使用 FPGA 的推断加速器，通过优化计算数据流、降低存储需求和优化卷积神经网络的部署，实现了支持任意核大小的卷积神经网络的高效部署，从而在各种基于视觉的应用中取得了卓越的性能。