The recent advancements in small-size inference models facilitated AI
deployment on the edge. However, the limited resource nature of edge devices
poses new challenges especially for real-time applications. Deploying multiple
inference models (or a single tunable model) varying in size and therefore
accuracy and power consumption, in addition to an edge server inference model,
can offer a dynamic system in which the allocation of inference models to
inference jobs is performed according to the current resource conditions.
Therefore, in this work, we tackle the problem of selectively allocating
inference models to jobs or offloading them to the edge server to maximize
inference accuracy under time and energy constraints. This problem is shown to
be an instance of the unbounded multidimensional knapsack problem which is
considered a strongly NP-hard problem. We propose a lightweight hybrid genetic
algorithm (LGSTO) to solve this problem. We introduce a termination condition
and neighborhood exploration techniques for faster evolution of populations. We
compare LGSTO with the Naive and Dynamic programming solutions. In addition to
classic genetic algorithms using different reproduction methods including
NSGA-II, and finally we compare to other evolutionary methods such as Particle
swarm optimization (PSO) and Ant colony optimization (ACO). Experiment results
show that LGSTO performed 3 times faster than the fastest comparable schemes
while producing schedules with higher average accuracy.

通过提出轻量级混合遗传算法来解决小型推理模型在边缘设备上动态分配推理任务以最大化推理准确度的问题，并与其他演化方法进行比较。