BriefGPT.xyz
Feb, 2025
条件激活神经网络的光线追踪
Ray-Tracing for Conditionally Activated Neural Networks
HTML
PDF
Claudio Gallicchio, Giuseppe Nuti
TL;DR
本文提出了一种新颖的条件激活神经网络架构，结合了多层次混合专家（MoEs）构造与逐步收敛的采样机制，解决了网络结构动态展开的问题。实验结果表明，该方法在保持竞争性准确率的同时显著减少了推理所需的参数数量，且这一参数减少与输入模式的复杂性相关，无需额外的惩罚函数。
Abstract
In this paper, we introduce a novel architecture for conditionally activated
Neural Networks
combining a hierarchical construction of multiple
Mixture of Experts
(MoEs) layers with a sampling mechanism that progr
→