Imaging sites around the world generate growing amounts of medical scan data
with ever more versatile and affordable technology. Large-scale studies acquire
MRI for tens of thousands of participants, together with metadata ranging from
lifestyle questionnaires to biochemical assays, genetic analyses and more.
These large datasets encode substantial information about human health and hold
considerable potential for machine learning training and analysis. This chapter
examines ongoing large-scale studies and the challenge of distribution shifts
between them. Transfer learning for overcoming such shifts is discussed,
together with federated learning for safe access to distributed training data
securely held at multiple institutions. Finally, representation learning is
reviewed as a methodology for encoding embeddings that express abstract
relationships in multi-modal input formats.

医学扫描数据、大规模研究、机器学习训练、分布转变和表征学习是本研究论文的主要关键词和主题。

MRI 数据处理的机器学习技术在不断扩展的规模下的应用

Machine Learning Techniques for MRI Data Processing at Expanding Scale

The input data pipeline is an essential component of each machine learning
(ML) training job. It is responsible for reading massive amounts of training
data, processing batches of samples using complex of transformations, and
loading them onto training nodes at low latency and high throughput. Performant
input data systems are becoming increasingly critical, driven by skyrocketing
data volumes and training throughput demands. Unfortunately, current input data
systems cannot fully leverage key performance optimizations, resulting in
hugely inefficient infrastructures that require significant resources -- or
worse -- underutilize expensive accelerators.
To address these demands, we present cedar, a programming model and framework
that allows users to easily build, optimize, and execute input data pipelines.
cedar presents an easy-to-use programming interface, allowing users to define
input data pipelines using composable operators that support arbitrary ML
frameworks and libraries. Meanwhile, cedar transparently applies a complex and
extensible set of optimization techniques (e.g., offloading, caching,
prefetching, fusion, and reordering). It then orchestrates processing across a
customizable set of local and distributed compute resources in order to
maximize processing performance and efficiency, all without user input. On
average across six diverse input data pipelines, cedar achieves a 2.49x, 1.87x,
2.18x, and 2.74x higher performance compared to tf.data, tf.data service, Ray
Data, and PyTorch's DataLoader, respectively.

cedar 是一个编程模型和框架，允许用户轻松构建、优化和执行输入数据管道，并在六个不同输入数据管道中与 tf.data、tf.data 服务、Ray Data 和 PyTorch 的 DataLoader 相比，平均获得 2.49 倍、1.87 倍、2.18 倍和 2.74 倍更高的性能。

cedar：可组合和优化的机器学习输入数据流程

cedar: Composable and Optimized Machine Learning Input Data Pipelines

Recent breakthroughs in generative artificial intelligence have triggered a
surge in demand for machine learning training, which poses significant cost
burdens and environmental challenges due to its substantial energy consumption.
Scheduling training jobs among geographically distributed cloud data centers
unveils the opportunity to optimize the usage of computing capacity powered by
inexpensive and low-carbon energy and address the issue of workload imbalance.
To tackle the challenge of multi-objective scheduling, i.e., maximizing GPU
utilization while reducing operational costs, we propose an algorithm based on
multi-agent reinforcement learning and actor-critic methods to learn the
optimal collaborative scheduling strategy through interacting with a cloud
system built with real-life workload patterns, energy prices, and carbon
intensities. Compared with other algorithms, our proposed method improves the
system utility by up to 28.6% attributable to higher GPU utilization, lower
energy cost, and less carbon emission.

本研究提出一种基于多智能体强化学习和演员 - 评论家方法的算法，通过与真实工作负载模式、能源价格和碳强度相互作用的云系统学习最优的协同调度策略，从而最大化 GPU 利用率、降低运营成本和减少碳排放。与其他算法相比，我们的方法提高了系统效用，最高可达 28.6％。

基于多智能体强化学习的地理分布数据中心可持续负载调度

Sustainable AIGC Workload Scheduling of Geo-Distributed Data Centers: A  Multi-Agent Reinforcement Learning Approach

Federated learning (FL) and split learning (SL) are state-of-the-art
distributed machine learning techniques to enable machine learning training
without accessing raw data on clients or end devices. However, their
\emph{comparative training performance} under real-world resource-restricted
Internet of Things (IoT) device settings, e.g., Raspberry Pi, remains barely
studied, which, to our knowledge, have not yet been evaluated and compared,
rendering inconvenient reference for practitioners. This work firstly provides
empirical comparisons of FL and SL in real-world IoT settings regarding (i)
learning performance with heterogeneous data distributions and (ii) on-device
execution overhead. Our analyses in this work demonstrate that the learning
performance of SL is better than FL under an imbalanced data distribution but
worse than FL under an extreme non-IID data distribution. Recently, FL and SL
are combined to form splitfed learning (SFL) to leverage each of their benefits
(e.g., parallel training of FL and lightweight on-device computation
requirement of SL). This work then considers FL, SL, and SFL, and mount them on
Raspberry Pi devices to evaluate their performance, including training time,
communication overhead, power consumption, and memory usage. Besides
evaluations, we apply two optimizations. Firstly, we generalize SFL by
carefully examining the possibility of a hybrid type of model training at the
server-side. The generalized SFL merges sequential (dependent) and parallel
(independent) processes of model training and is thus beneficial for a system
with large-scaled IoT devices, specifically at the server-side operations.
Secondly, we propose pragmatic techniques to substantially reduce the
communication overhead by up to four times for the SL and (generalized) SFL.

该论文在实际物联网设置中比较了联邦学习、分裂学习和分裂联邦学习的性能，并提出了一种广义分裂联邦学习模型来进一步优化在大规模 IoT 设备上的系统表现，同时还提出实用技巧来大幅减少通信开销。