Large language models (LLMs) have become a significant workload since their
appearance. However, they are also computationally expensive as they have
billions of parameters and are trained with massive amounts of data. Thus,
recent works have developed dedicated CUDA kernels for LLM training and
inference instead of relying on compilergenerated ones, so that hardware
resources are as fully utilized as possible. In this work, we explore the
possibility of GPU native instruction optimization to further push the CUDA
kernels to extreme performance. Contrary to prior works, we adopt an automatic
optimization approach by defining a search space of possible GPU native
instruction schedules, and then we apply stochastic search to perform
optimization. Experiments show that SIP can further improve CUDA kernel
throughput by automatically discovering better GPU native instruction schedules
and the optimized schedules are tested by 10 million test samples.

通过自动优化搜索空间和随机搜索方法，我们实现了 GPU 原生指令优化，从而进一步提高了 CUDA 核心的吞吐量，并通过 1000 万个测试样本对优化后的指令进行了测试。

SIP: 通过随机指令摄动自动调优 GPU 本地调度

SIP: Autotuning GPU Native Schedules via Stochastic Instruction  Perturbation

Windowing is a common technique in EEG machine learning classification and
other time series tasks. However, a challenge arises when employing this
technique: computational expense inhibits learning global relationships across
an entire recording or set of recordings. Furthermore, the labels inherited by
windows from their parent recordings may not accurately reflect the content of
that window in isolation. To resolve these issues, we introduce a multi-stage
model architecture, incorporating meta-learning principles tailored to
time-windowed data aggregation. We further tested two distinct strategies to
alleviate these issues: lengthening the window and utilizing overlapping to
augment data. Our methods, when tested on the Temple University Hospital
Abnormal EEG Corpus (TUAB), dramatically boosted the benchmark accuracy from
89.8 percent to 99.0 percent. This breakthrough performance surpasses prior
performance projections for this dataset and paves the way for clinical
applications of machine learning solutions to EEG interpretation challenges. On
a broader and more varied dataset from the Temple University Hospital EEG
Corpus (TUEG), we attained an accuracy of 86.7%, nearing the assumed
performance ceiling set by variable inter-rater agreement on such datasets.

通过引入多阶段模型架构和元学习原理，结合窗体化的数据聚合方法，我们成功解决了 EEG 机器学习分类和其他时间序列任务中遇到的窗体计算困难、数据标签不准确等问题，并在 Temple 大学医院异常 EEG 数据库（TUAB）上将基准准确率从 89.8% 大幅提升至 99.0%，为机器学习在 EEG 解读挑战中的临床应用提供了突破性的解决方案。在 Temple 大学医院 EEG 数据库（TUEG）上，我们实现了 86.7% 的准确率，接近变量评价者一致性设定的数据性能上限。

窗口叠加元模型用于临床脑电分类

Window Stacking Meta-Models for Clinical EEG Classification

One-shot Neural Architecture Search (NAS) aims to minimize the computational
expense of discovering state-of-the-art models. However, in the past year
attention has been drawn to the comparable performance of naive random search
across the same search spaces used by leading NAS algorithms. To address this,
we explore the effects of drastically relaxing the NAS search space, and we
present Bonsai-Net, an efficient one-shot NAS method to explore our relaxed
search space. Bonsai-Net is built around a modified differential pruner and can
consistently discover state-of-the-art architectures that are significantly
better than random search with fewer parameters than other state-of-the-art
methods. Additionally, Bonsai-Net performs simultaneous model search and
training, dramatically reducing the total time it takes to generate
fully-trained models from scratch.

通过放宽神经架构搜索领域的范围，本研究提出了一种高效的一次性神经架构搜索方法 Bonsai-Net，可同时进行模型搜索和训练，并可显著减少使用更少参数比其他现有方法更好地发现优秀架构的全过程所需的时间。