AI tasks encompass a wide range of domains and fields. While numerous AI
models have been designed for specific tasks and applications, they often
require considerable human efforts in finding the right model architecture,
optimization algorithm, and hyperparameters. Recent advances in large language
models (LLMs) like ChatGPT show remarkable capabilities in various aspects of
reasoning, comprehension, and interaction. Consequently, we propose developing
task-oriented prompts and automatically utilizing LLMs to automate the training
pipeline. To implement this concept, we present the AutoML-GPT, which employs
GPT as the bridge to diverse AI models and dynamically trains models with
optimized hyperparameters. AutoML-GPT dynamically takes user requests from the
model and data cards and composes the corresponding prompt paragraph.
Ultimately, with this prompt paragraph, AutoML-GPT will automatically conduct
the experiments from data processing to model architecture, hyperparameter
tuning, and predicted training log. By leveraging {\ours}'s robust language
capabilities and the available AI models, AutoML-GPT can tackle numerous
intricate AI tasks across various tasks and datasets. This approach achieves
remarkable results in computer vision, natural language processing, and other
challenging areas. Extensive experiments and ablation studies demonstrate that
our method can be general, effective, and beneficial for many AI tasks.

AutoML-GPT 是一种使用大型语言模型和任务导向提示的自动化训练管道，可自动处理数据、选择最佳模型架构和超参数优化，并在计算机视觉、自然语言处理等领域实现了显著的结果，对于许多 AI 任务具有广泛的适用性。

AutoML-GPT：使用 GPT 进行自动化机器学习

AutoML-GPT: Automatic Machine Learning with GPT

Automating video-based data and machine learning pipelines poses several
challenges including metadata generation for efficient storage and retrieval
and isolation of key-frames for scene understanding tasks. In this work, we
present two semi-supervised approaches that automate this process of manual
frame sifting in video streams by automatically classifying scenes for content
and filtering frames for fine-tuning scene understanding tasks. The first
rule-based method starts from a pre-trained object detector and it assigns
scene type, uncertainty and lighting categories to each frame based on
probability distributions of foreground objects. Next, frames with the highest
uncertainty and structural dissimilarity are isolated as key-frames. The second
method relies on the simCLR model for frame encoding followed by
label-spreading from 20% of frame samples to label the remaining frames for
scene and lighting categories. Also, clustering the video frames in the encoded
feature space further isolates key-frames at cluster boundaries. The proposed
methods achieve 64-93% accuracy for automated scene categorization for outdoor
image videos from public domain datasets of JAAD and KITTI. Also, less than 10%
of all input frames can be filtered as key-frames that can then be sent for
annotation and fine tuning of machine vision algorithms. Thus, the proposed
framework can be scaled to additional video data streams for automated training
of perception-driven systems with minimal training images.

本文提出了两种半监督方法以自动化视频流中的场景分类和筛选关键帧。所提出的框架可用于增加的视频数据流，以最少的训练图像自动训练感知驱动系统。