Evolutionary algorithms are increasingly recognised as a viable computational
approach for the automated optimisation of deep neural networks (DNNs) within
artificial intelligence. This method extends to the training of DNNs, an
approach known as neuroevolution. However, neuroevolution is an inherently
resource-intensive process, with certain studies reporting the consumption of
thousands of GPU days for refining and training a single DNN network. To
address the computational challenges associated with neuroevolution while still
attaining good DNN accuracy, surrogate models emerge as a pragmatic solution.
Despite their potential, the integration of surrogate models into
neuroevolution is still in its early stages, hindered by factors such as the
effective use of high-dimensional data and the representation employed in
neuroevolution. In this context, we address these challenges by employing a
suitable representation based on Linear Genetic Programming, denoted as
NeuroLGP, and leveraging Kriging Partial Least Squares. The amalgamation of
these two techniques culminates in our proposed methodology known as the
NeuroLGP-Surrogate Model (NeuroLGP-SM). For comparison purposes, we also code
and use a baseline approach incorporating a repair mechanism, a common practice
in neuroevolution. Notably, the baseline approach surpasses the renowned VGG-16
model in accuracy. Given the computational intensity inherent in DNN
operations, a singular run is typically the norm. To evaluate the efficacy of
our proposed approach, we conducted 96 independent runs. Significantly, our
methodologies consistently outperform the baseline, with the SM model
demonstrating superior accuracy or comparable results to the NeuroLGP approach.
Noteworthy is the additional advantage that the SM approach exhibits a 25%
reduction in computational requirements, further emphasising its efficiency for
neuroevolution.

我们通过采用基于线性遗传编程的适当表示方法（称为 NeuroLGP）和应用克里金偏最小二乘法来解决神经演化中的计算挑战，提出了一种称为 NeuroLGP-Surrogate Model（NeuroLGP-SM）的方法。该方法在 96 次独立运行中表现出卓越的精度，并且比基线方法节省了 25% 的计算资源，进一步突显了其在神经演化中的效率。

神经 LGP-SM：基于线性遗传编程的辅助神经进化方法

NeuroLGP-SM: A Surrogate-assisted Neuroevolution Approach using Linear  Genetic Programming

Genetic improvement is a search technique that aims to improve a given
acceptable solution to a problem. In this paper, we present the novel use of
genetic improvement to find problem-specific optimized LLVM pass sequences. We
develop a pass-level patch representation in the linear genetic programming
framework, Shackleton, to evolve the modifications to be applied to the default
optimization pass sequences. Our GI-evolved solution has a mean of 3.7% runtime
improvement compared to the -O3 optimization level in the default code
generation options which optimizes on runtime. The proposed GI method provides
an automatic way to find a problem-specific optimization sequence that improves
upon a general solution without any expert domain knowledge. In this paper, we
discuss the advantages and limitations of the GI feature in the Shackleton
Framework and present our results.

本文介绍了一种新颖的基因改进技术，用于寻找特定问题的优化 LLVM 传递序列，结果表明此方法具有自动、无需领域专家知识的特点，并成功地实现了优化结果的改进。

优化 LLVM Pass sequences 的 Shackleton 框架的遗传改进

Genetic Improvement in the Shackleton Framework for Optimizing LLVM Pass Sequences

In this paper we introduce Shackleton as a generalized framework enabling the
application of linear genetic programming -- a technique under the umbrella of
evolutionary algorithms -- to a variety of use cases. We also explore here a
novel application for this class of methods: optimizing sequences of LLVM
optimization passes. The algorithm underpinning Shackleton is discussed, with
an emphasis on the effects of different features unique to the framework when
applied to LLVM pass sequences. Combined with analysis of different
hyperparameter settings, we report the results on automatically optimizing pass
sequences using Shackleton for two software applications at differing
complexity levels. Finally, we reflect on the advantages and limitations of our
current implementation and lay out a path for further improvements. These
improvements aim to surpass hand-crafted solutions with an automatic discovery
method for an optimal pass sequence.

本研究介绍了 Shackleton 框架，将线性遗传编程技术应用于 LLVM 优化传递的序列优化，实现自动化优化；结果表明，自动优化可以优于手工优化方案。