We break the linear link between the layer size and its inference cost by
introducing the fast feedforward (FFF) architecture, a logarithmic-time
alternative to feedforward networks.
We show that FFFs give comparable performance to feedforward networks at an
exponential fraction of their inference cost, are quicker to deliver
performance compared to mixture-of-expert networks, and can readily take the
place of either in transformers.
Pushing FFFs to the absolute limit, we train a vision transformer to perform
single-neuron inferences at the cost of only 5.8% performance decrease against
the full-width variant.
Our implementation is available as a Python package; just use "pip install
fastfeedforward".

我们介绍了快速前馈（FFF）架构，它打破了层大小与推理成本之间的线性关系，通过引入对数时间的前馈网络的替代方案。我们展示了 FFF 在推理成本的指数分数上表现出与前馈网络相当的性能，与专家混合网络相比更快地提供性能，并且可以在 transformers 中取代它们。将 FFF 推到极限，我们训练了一个视觉 transformer，在只有 5.8% 性能下降的成本下执行单神经元推理。我们的实现可作为 Python 包使用，只需运行 “pip install fastfeedforward” 即可。