In the realm of machine learning, traditional model development and automated
approaches like AutoML typically rely on layers of abstraction, such as
tree-based or Cartesian genetic programming. Our study introduces "Guided
Evolution" (GE), a novel framework that diverges from these methods by
utilizing Large Language Models (LLMs) to directly modify code. GE leverages
LLMs for a more intelligent, supervised evolutionary process, guiding mutations
and crossovers. Our unique "Evolution of Thought" (EoT) technique further
enhances GE by enabling LLMs to reflect on and learn from the outcomes of
previous mutations. This results in a self-sustaining feedback loop that
augments decision-making in model evolution. GE maintains genetic diversity,
crucial for evolutionary algorithms, by leveraging LLMs' capability to generate
diverse responses from expertly crafted prompts and modulate model temperature.
This not only accelerates the evolution process but also injects expert like
creativity and insight into the process. Our application of GE in evolving the
ExquisiteNetV2 model demonstrates its efficacy: the LLM-driven GE autonomously
produced variants with improved accuracy, increasing from 92.52% to 93.34%,
without compromising model compactness. This underscores the potential of LLMs
to accelerate the traditional model design pipeline, enabling models to
autonomously evolve and enhance their own designs.

通过利用大型语言模型的指导进化框架，研究证明了机器学习模型的自主进化和增强设计的潜力。

LLM 引导演化：模型升级的自动化

LLM Guided Evolution - The Automation of Models Advancing Models

How to automatically design better machine learning programs is an open
problem within AutoML. While evolution has been a popular tool to search for
better ML programs, using learning itself to guide the search has been less
successful and less understood on harder problems but has the promise to
dramatically increase the speed and final performance of the optimization
process. We propose guiding evolution with a binary discriminator, trained
online to distinguish which program is better given a pair of programs. The
discriminator selects better programs without having to perform a costly
evaluation and thus speed up the convergence of evolution. Our method can
encode a wide variety of ML components including symbolic optimizers, neural
architectures, RL loss functions, and symbolic regression equations with the
same directed acyclic graph representation. By combining this representation
with modern GNNs and an adaptive mutation strategy, we demonstrate our method
can speed up evolution across a set of diverse problems including a 3.7x
speedup on the symbolic search for ML optimizers and a 4x speedup for RL loss
functions.

使用二元辨别器进行进化引导可以加速演化过程并解决自动设计机器学习程序的问题。