Today's deep learning methods focus on how to design the most appropriate
objective functions so that the prediction results of the model can be closest
to the ground truth. Meanwhile, an appropriate architecture that can facilitate
acquisition of enough information for prediction has to be designed. Existing
methods ignore a fact that when input data undergoes layer-by-layer feature
extraction and spatial transformation, large amount of information will be
lost. This paper will delve into the important issues of data loss when data is
transmitted through deep networks, namely information bottleneck and reversible
functions. We proposed the concept of programmable gradient information (PGI)
to cope with the various changes required by deep networks to achieve multiple
objectives. PGI can provide complete input information for the target task to
calculate objective function, so that reliable gradient information can be
obtained to update network weights. In addition, a new lightweight network
architecture -- Generalized Efficient Layer Aggregation Network (GELAN), based
on gradient path planning is designed. GELAN's architecture confirms that PGI
has gained superior results on lightweight models. We verified the proposed
GELAN and PGI on MS COCO dataset based object detection. The results show that
GELAN only uses conventional convolution operators to achieve better parameter
utilization than the state-of-the-art methods developed based on depth-wise
convolution. PGI can be used for variety of models from lightweight to large.
It can be used to obtain complete information, so that train-from-scratch
models can achieve better results than state-of-the-art models pre-trained
using large datasets, the comparison results are shown in Figure 1. The source
codes are at: this https URL

当前的深度学习方法主要关注如何设计最合适的目标函数，以使得模型的预测结果最接近真实情况，同时需要设计一种适当的架构来获取足够的信息进行预测。然而现有的方法忽视了一个事实，即当输入数据经历逐层特征提取和空间变换时，会丢失大量的信息。本文主要研究数据在通过深度网络传输时的信息损失问题，即信息瓶颈和可逆函数。我们提出了可编程梯度信息（PGI）的概念，以应对深度网络需求多样的变化来实现多个目标。PGI 能够为目标任务提供完整的输入信息来计算目标函数，从而获取可靠的梯度信息来更新网络权重。此外，我们设计了一种新的轻量级网络架构 —— 广义高效层聚合网络（GELAN），基于梯度路径规划。GELAN 的架构证明了在轻量级模型上 PGI 取得了更好的结果。我们在基于 MS COCO 数据集的目标检测任务上验证了 GELAN 和 PGI，结果显示 GELAN 只使用传统的卷积算子就能比基于深度可分离卷积的最新方法实现更好的参数利用率。PGI 可用于各种模型，从轻量级到大型模型。它可以获取完整信息，从而使得从头开始训练的模型比使用大型数据集预训练的最新模型获得更好的结果，对比结果详见图 1。源代码位于：[指定的 URL]