Convolutional Neural Networks (CNNs) have demonstrated their effectiveness in
numerous vision tasks. However, their high processing requirements necessitate
efficient hardware acceleration to meet the application's performance targets.
In the space of FPGAs, streaming-based dataflow architectures are often adopted
by users, as significant performance gains can be achieved through layer-wise
pipelining and reduced off-chip memory access by retaining data on-chip.
However, modern topologies, such as the UNet, YOLO, and X3D models, utilise
long skip connections, requiring significant on-chip storage and thus limiting
the performance achieved by such system architectures. The paper addresses the
above limitation by introducing weight and activation eviction mechanisms to
off-chip memory along the computational pipeline, taking into account the
available compute and memory resources. The proposed mechanism is incorporated
into an existing toolflow, expanding the design space by utilising off-chip
memory as a buffer. This enables the mapping of such modern CNNs to devices
with limited on-chip memory, under the streaming architecture design approach.
SMOF has demonstrated the capacity to deliver competitive and, in some cases,
state-of-the-art performance across a spectrum of computer vision tasks,
achieving up to 10.65 X throughput improvement compared to previous works.

该论文介绍了一种在计算流水线中引入权重和激活记忆机制的方法，从而扩展了基于流数据架构设计方法中的设计空间，实现了将现代卷积神经网络映射到具有有限芯片内存的设备上。此方法通过在计算流水线中将权重和激活数据驱逐到芯片外存储器中，充分利用计算和内存资源。通过该方法，该论文证明其在计算机视觉任务中具有竞争力，甚至在某些情况下表现出最先进的性能，较以前的方法提高了高达 10.65 倍的吞吐量。