Capsule Networks (CapsNet) use the Softmax function to convert the logits of
the routing coefficients into a set of normalized values that signify the
assignment probabilities between capsules in adjacent layers. We show that the
use of Softmax prevents capsule layers from forming optimal couplings between
lower and higher-level capsules. Softmax constrains the dynamic range of the
routing coefficients and leads to probabilities that remain mostly uniform
after several routing iterations. Instead, we propose the use of Max-Min
normalization. Max-Min performs a scale-invariant normalization of the logits
that allows each lower-level capsule to take on an independent value,
constrained only by the bounds of normalization. Max-Min provides consistent
improvement in test accuracy across five datasets and allows more routing
iterations without a decrease in network performance. A single CapsNet trained
using Max-Min achieves an improved test error of 0.20% on the MNIST dataset.
With a simple 3-model majority vote, we achieve a test error of 0.17% on MNIST.

本文研究了 Capsule Networks 中的 Softmax Function 在正常化路由系数时存在的问题，并提出了 Max-Min 正常化方法。经实验证明，使用 Max-Min 方法能够在提高测试准确性的同时，允许更多路由次数而不会降低网络性能。