With more scientific fields relying on neural networks (NNs) to process data incoming at extreme throughputs and latencies, it is crucial to develop NNs with all their parameters stored on-chip. In many of these applications, there is not enough time to go off-chip and retrieve weights. Even more so, off-chip memory such as DRAM does not have the bandwidth required to process these NNs as fast as the data is being produced (e.g., every 25 ns). As such, these extreme latency and bandwidth requirements have architectural implications for the hardware intended to run these NNs: 1) all NN parameters must fit on-chip, and 2) codesigning custom/reconfigurable logic is often required to meet these latency and bandwidth constraints. In our work, we show that many scientific NN applications must run fully on chip, in the extreme case requiring a custom chip to meet such stringent constraints.

科学领域越来越依赖于神经网络（NN）来处理大量数据的传入，而且延迟也很高，因此开发具有存储在芯片上的所有参数的NN非常关键。我们的研究表明，许多科学NN应用必须完全在芯片上运行，极端情况下需要自定制芯片来满足严格的约束条件。

神经网络推理对高数据速率、低延迟科学应用的架构影响