Recently, Tensor Ring Networks (TRNs) have been applied in deep networks,
achieving remarkable successes in compression ratio and accuracy. Although
highly related to the performance of TRNs, rank selection is seldom studied in
previous works and usually set to equal in experiments. Meanwhile, there is not
any heuristic method to choose the rank, and an enumerating way to find
appropriate rank is extremely time-consuming. Interestingly, we discover that
part of the rank elements is sensitive and usually aggregate in a narrow
region, namely an interest region. Therefore, based on the above phenomenon, we
propose a novel progressive genetic algorithm named Progressively Searching
Tensor Ring Network Search (PSTRN), which has the ability to find optimal rank
precisely and efficiently. Through the evolutionary phase and progressive
phase, PSTRN can converge to the interest region quickly and harvest good
performance. Experimental results show that PSTRN can significantly reduce the
complexity of seeking rank, compared with the enumerating method. Furthermore,
our method is validated on public benchmarks like MNIST, CIFAR10/100, UCF11 and
HMDB51, achieving the state-of-the-art performance.

本文研究了 Tensor Ring Networks 在深度神经网络中的应用，发现前人研究中很少涉及其等级选择问题。在此基础上，作者提出了一种名为 PSTRN 的新颖渐进遗传算法，能有效寻找最优等级，实验证明该算法可以显著降低寻找等级的复杂度，并在 MNIST、CIFAR10/100、UCF11 和 HMDB51 等公共基准测试中实现了最先进的性能。

使用渐进式搜索张量环网络进行启发式排名选择

Heuristic Rank Selection with Progressively Searching Tensor Ring  Network

Low-rank decomposition plays a central role in accelerating convolutional
neural network (CNN), and the rank of decomposed kernel-tensor is a key
parameter that determines the complexity and accuracy of a neural network. In
this paper, we define rank selection as a combinatorial optimization problem
and propose a methodology to minimize network complexity while maintaining the
desired accuracy. Combinatorial optimization is not feasible due to search
space limitations. To restrict the search space and obtain the optimal rank, we
define the space constraint parameters with a boundary condition. We also
propose a linearly-approximated accuracy function to predict the fine-tuned
accuracy of the optimized CNN model during the cost reduction. Experimental
results on AlexNet and VGG-16 show that the proposed rank selection algorithm
satisfies the accuracy constraint. Our method combined with truncated-SVD
outperforms state-of-the-art methods in terms of inference and training time at
almost the same accuracy.

本文提出了一种通过组合优化实现网络复杂度最小化以及保持精度不变的方案，利用线性近似的精度函数预测了优化后 CNN 模型的精度，并在 AlexNet 和 VGG-16 上的实验结果表明，与截断 SVD 算法相结合，我们提出的排名选择算法在推理和训练时间方面优于现有技术。