In most applications of utilizing neural networks for mathematical optimization, a dedicated model is trained for each specific optimization objective. However, in many scenarios, several distinct yet correlated objectives or tasks often need to be optimized on the same set of problem inputs. Instead of independently training a different neural network for each problem separately, it would be more efficient to exploit the correlations between these objectives and to train multiple neural network models with shared model parameters and feature representations. To achieve this, this paper first establishes the concept of common information: the shared knowledge required for solving the correlated tasks, then proposes a novel approach for model training by adding into the model an additional reconstruction stage associated with a new reconstruction loss. This loss is for reconstructing the common information starting from a selected hidden layer in the model. The proposed approach encourages the learned features to be general and transferable, and therefore can be readily used for efficient transfer learning. For numerical simulations, three applications are studied: transfer learning on classifying MNIST handwritten digits, the device-to-device wireless network power allocation, and the multiple-input-single-output network downlink beamforming and localization. Simulation results suggest that the proposed approach is highly efficient in data and model complexity, is resilient to over-fitting, and has competitive performances.

该论文提出了一种利用神经网络进行优化的方法，该方法通过共享模型参数和特征表示来训练多个神经网络模型，以解决多个相关任务。论文引入了共同信息的概念，并提出了一种新的模型训练方法，通过添加一个额外的重构阶段和重构损失，将共同信息重新构建出来。该方法鼓励学习到的特征具有通用性和可迁移性，因此可以用于有效的迁移学习。通过数值模拟，研究了在不同应用领域上的性能表现。结果表明，该方法在数据和模型复杂度方面非常高效，能够抵抗过拟合，并且具有竞争性能。

带有重建损失的迁移学习