GPU-embedded systems have gained popularity across various domains due to
their efficient power consumption. However, in order to meet the demands of
real-time or time-consuming applications running on these systems, it is
crucial for them to be tuned to exhibit high performance. This paper addresses
the issue by developing and comparing two tuning methodologies on GPU-embedded
systems, and also provides performance insights for developers and researchers
seeking to optimize applications running on these architectures. We focus on
parallel prefix operations, such as FFT, scan primitives, and tridiagonal
system solvers, which are performance-critical components in many applications.
The study introduces an analytical model-driven tuning methodology and a
Machine Learning (ML)-based tuning methodology. We evaluate the performance of
the two tuning methodologies for different parallel prefix implementations of
the BPLG library in an NVIDIA Jetson system, and compare their performance to
the ones achieved through an exhaustive search. The findings shed light on the
best strategies for handling the open challenge of performance portability for
major computational patterns among server and embedded devices, providing
practical guidance for offline and online tuning. We also address the existing
gap in performance studies for parallel computational patterns in GPU-embedded
systems by comparing the BPLG performance against other state-of-the-art
libraries, including CUSPARSE, CUB, and CUFFT.

本论文研究了 GPU 嵌入式系统上的调优方法，并比较了基于分析模型和基于机器学习的调优方法的性能，在 NVIDIA Jetson 系统上评估了 BPLG 库的不同并行前缀实现的性能，并与穷举搜索的性能进行了比较，为解决服务器和嵌入式设备间主要计算模式的性能可移植性提供了实用指导。

基于机器学习和分析模型的 GPU 嵌入式系统性能调优方法

Performance Tuning for GPU-Embedded Systems: Machine-Learning-based and  Analytical Model-driven Tuning Methodologies

Real-time semantic segmentation is a challenging task that requires
high-accuracy models with low-inference times. Implementing these models on
embedded systems is limited by hardware capability and memory usage, which
produces bottlenecks. We propose an efficient model for real-time semantic
segmentation called JetSeg, consisting of an encoder called JetNet, and an
improved RegSeg decoder. The JetNet is designed for GPU-Embedded Systems and
includes two main components: a new light-weight efficient block called
JetBlock, that reduces the number of parameters minimizing memory usage and
inference time without sacrificing accuracy; a new strategy that involves the
combination of asymmetric and non-asymmetric convolutions with
depthwise-dilated convolutions called JetConv, a channel shuffle operation,
light-weight activation functions, and a convenient number of group
convolutions for embedded systems, and an innovative loss function named
JetLoss, which integrates the Precision, Recall, and IoUB losses to improve
semantic segmentation and reduce computational complexity. Experiments
demonstrate that JetSeg is much faster on workstation devices and more suitable
for Low-Power GPU-Embedded Systems than existing state-of-the-art models for
real-time semantic segmentation. Our approach outperforms state-of-the-art
real-time encoder-decoder models by reducing 46.70M parameters and 5.14%
GFLOPs, which makes JetSeg up to 2x faster on the NVIDIA Titan RTX GPU and the
Jetson Xavier than other models. The JetSeg code is available at
this https URL

我们提出了一种用于实时语义分割的高效模型，称为 JetSeg，包括一个名为 JetNet 的编码器和一个改进的 RegSeg 解码器，JetNet 针对 GPU 嵌入式系统设计，包括 JetBlock 和 JetConv 两个主要组件以及 JetLoss 损失函数，实验结果表明，JetSeg 比现有的实时编码器 - 解码器模型更适合工作站设备和低功率 GPU 嵌入式系统，并且比其他模型快 2 倍。