Neural networks have been extensively applied to a variety of tasks,
achieving astounding results. Applying neural networks in the scientific field
is an important research direction that is gaining increasing attention. In
scientific applications, the scale of neural networks is generally
moderate-size, mainly to ensure the speed of inference during application.
Additionally, comparing neural networks to traditional algorithms in scientific
applications is inevitable. These applications often require rapid
computations, making the reduction of neural network sizes increasingly
important. Existing work has found that the powerful capabilities of neural
networks are primarily due to their non-linearity. Theoretical work has
discovered that under strong non-linearity, neurons in the same layer tend to
behave similarly, a phenomenon known as condensation. Condensation offers an
opportunity to reduce the scale of neural networks to a smaller subnetwork with
similar performance. In this article, we propose a condensation reduction
algorithm to verify the feasibility of this idea in practical problems. Our
reduction method can currently be applied to both fully connected networks and
convolutional networks, achieving positive results. In complex combustion
acceleration tasks, we reduced the size of the neural network to 41.7% of its
original scale while maintaining prediction accuracy. In the CIFAR10 image
classification task, we reduced the network size to 11.5% of the original
scale, still maintaining a satisfactory validation accuracy. Our method can be
applied to most trained neural networks, reducing computational pressure and
improving inference speed.

我们提出了一种凝结缩减算法，验证了这一想法在实际问题中的可行性，可以将神经网络的规模减小，并在保持预测准确性的同时减少计算压力和提高推理速度。

压缩中等规模深度神经网络的高效灵活方法

Efficient and Flexible Method for Reducing Moderate-size Deep Neural  Networks with Condensation

Materials datasets are usually featured by the existence of many redundant
(highly similar) materials due to the tinkering material design practice over
the history of materials research. For example, the materials project database
has many perovskite cubic structure materials similar to SrTiO$_3$. This sample
redundancy within the dataset makes the random splitting of machine learning
model evaluation to fail so that the ML models tend to achieve over-estimated
predictive performance which is misleading for the materials science community.
This issue is well known in the field of bioinformatics for protein function
prediction, in which a redundancy reduction procedure (CD-Hit) is always
applied to reduce the sample redundancy by ensuring no pair of samples has a
sequence similarity greater than a given threshold. This paper surveys the
overestimated ML performance in the literature for both composition based and
structure based material property prediction. We then propose a material
dataset redundancy reduction algorithm called MD-HIT and evaluate it with
several composition and structure based distance threshold sfor reducing data
set sample redundancy. We show that with this control, the predicted
performance tends to better reflect their true prediction capability. Our
MD-hit code can be freely accessed at this https URL

本文调查了文献中关于基于组成和基于结构的材料属性预测的过高 ML 性能，提出了一种材料数据集冗余降低算法 MD-HIT，并通过几个组成和结构基础的距离阈值评估其性能，结果表明，通过这种方法，预测性能更能反映其真实的预测能力。