We introduce latency-aware network acceleration (LANA) - an approach that
builds on neural architecture search techniques and teacher-student
distillation to accelerate neural networks. LANA consists of two phases: in the
first phase, it trains many alternative operations for every layer of the
teacher network using layer-wise feature map distillation. In the second phase,
it solves the combinatorial selection of efficient operations using a novel
constrained integer linear optimization (ILP) approach. ILP brings unique
properties as it (i) performs NAS within a few seconds to minutes, (ii) easily
satisfies budget constraints, (iii) works on the layer-granularity, (iv)
supports a huge search space $O(10^{100})$, surpassing prior search approaches
in efficacy and efficiency. In extensive experiments, we show that LANA yields
efficient and accurate models constrained by a target latency budget, while
being significantly faster than other techniques. We analyze three popular
network architectures: EfficientNetV1, EfficientNetV2 and ResNeST, and achieve
accuracy improvement for all models (up to $3.0\%$) when compressing larger
models to the latency level of smaller models. LANA achieves significant
speed-ups (up to $5\times$) with minor to no accuracy drop on GPU and CPU. The
code will be shared soon.

介绍了一种基于神经结构搜索技术和教师 - 学生知识蒸馏的延迟感知网络加速（LANA）方法，它在实验中取得了优异的效果，大幅提升了神经网络加速的速度和准确性。

LANA: 延迟感知网络加速

LANA: Latency Aware Network Acceleration

We propose an abstraction-based multi-document summarization framework that
can construct new sentences by exploring more fine-grained syntactic units than
sentences, namely, noun/verb phrases. Different from existing abstraction-based
approaches, our method first constructs a pool of concepts and facts
represented by phrases from the input documents. Then new sentences are
generated by selecting and merging informative phrases to maximize the salience
of phrases and meanwhile satisfy the sentence construction constraints. We
employ integer linear optimization for conducting phrase selection and merging
simultaneously in order to achieve the global optimal solution for a summary.
Experimental results on the benchmark data set TAC 2011 show that our framework
outperforms the state-of-the-art models under automated pyramid evaluation
metric, and achieves reasonably well results on manual linguistic quality
evaluation.

我们提出了一种基于抽象的多文档摘要框架，可以通过探索比句子更细粒度的句法单元（即名词 / 动词短语）来构建新的句子。与现有的基于抽象的方法不同，我们的方法首先构建了一个由输入文档中的短语表示的概念和事实池。然后通过选择和合并信息短语生成新的句子，以最大化短语的显著性并同时满足句子构造约束。我们采用整数线性优化来同时进行短语选择和合并，以实现摘要的全局最优解。在基准数据集 TAC 2011 上进行的实验结果表明，我们的框架在自动金字塔评估度量标准下优于最先进的模型，并在手动语言质量评估上取得了相当好的结果。