Deep neural networks (DNNs) have become ubiquitous in addressing a number of
problems, particularly in computer vision. However, DNN inference is
computationally intensive, which can be prohibitive e.g. when considering edge
devices. To solve this problem, a popular solution is DNN pruning, and more so
structured pruning, where coherent computational blocks (e.g. channels for
convolutional networks) are removed: as an exhaustive search of the space of
pruned sub-models is intractable in practice, channels are typically removed
iteratively based on an importance estimation heuristic. Recently, promising
latency-aware pruning methods were proposed, where channels are removed until
the network reaches a target budget of wall-clock latency pre-emptively
estimated on specific hardware. In this paper, we present Archtree, a novel
method for latency-driven structured pruning of DNNs. Archtree explores
multiple candidate pruned sub-models in parallel in a tree-like fashion,
allowing for a better exploration of the search space. Furthermore, it involves
on-the-fly latency estimation on the target hardware, accounting for closer
latencies as compared to the specified budget. Empirical results on several DNN
architectures and target hardware show that Archtree better preserves the
original model accuracy while better fitting the latency budget as compared to
existing state-of-the-art methods.

Archtree 是一种新的基于延迟驱动的 DNN 结构修剪方法，通过并行地在树形结构中探索多个候选修剪子模型，实时估计目标硬件的延迟，从而更好地适应延迟预算并保持原始模型准确性。