Capsule Networks, an extension to Neural Networks utilizing vector or matrix representations instead of scalars, were initially developed to create a dynamic parse tree where visual concepts evolve from parts to complete objects. Early implementations of Capsule Networks achieved and maintain state-of-the-art results on various datasets. However, recent studies have revealed shortcomings in the original Capsule Network architecture, notably its failure to construct a parse tree and its susceptibility to vanishing gradients when deployed in deeper networks. This paper extends the investigation to a range of leading Capsule Network architectures, demonstrating that these issues are not confined to the original design. We argue that the majority of Capsule Network research has produced architectures that, while modestly divergent from the original Capsule Network, still retain a fundamentally similar structure. We posit that this inherent design similarity might be impeding the scalability of Capsule Networks. Our study contributes to the broader discussion on improving the robustness and scalability of Capsule Networks.

本研究探讨了 Capsule Networks 神经网络的多个最新架构，发现构建解析树和在深层网络中可能出现梯度消失的问题不仅存在于最初的设计中，而且若干变体设计仍然保留了基本相似的结构，这种固有结构相似性可能妨碍了 Capsule Networks 的可扩展性。本研究有助于提高 Capsule Networks 的健壮性和可扩展性。

深度胶囊网络中的消失激活：一种症状