In recent years, software systems powered by deep learning (DL) techniques
have significantly facilitated people's lives in many aspects. As the backbone
of these DL systems, various DL libraries undertake the underlying optimization
and computation. However, like traditional software, DL libraries are not
immune to bugs, which can pose serious threats to users' personal property and
safety. Studying the characteristics of DL libraries, their associated bugs,
and the corresponding testing methods is crucial for enhancing the security of
DL systems and advancing the widespread application of DL technology. This
paper provides an overview of the testing research related to various DL
libraries, discusses the strengths and weaknesses of existing methods, and
provides guidance and reference for the application of the DL library. This
paper first introduces the workflow of DL underlying libraries and the
characteristics of three kinds of DL libraries involved, namely DL framework,
DL compiler, and DL hardware library. It then provides definitions for DL
underlying library bugs and testing. Additionally, this paper summarizes the
existing testing methods and tools tailored to these DL libraries separately
and analyzes their effectiveness and limitations. It also discusses the
existing challenges of DL library testing and outlines potential directions for
future research.

深度学习技术在各个领域极大地方便了人们的生活，但深度学习库作为这些系统的支柱，同样会受到漏洞的威胁。本文提供了与深度学习库相关的测试研究概述，讨论了现有方法的优势、劣势，并为深度学习库的应用提供指导和参考，同时还总结了针对这些库的现有测试方法和工具，分析了其有效性和局限性，并探讨了深度学习库测试的挑战和未来的研究方向。

深度学习库测试方法概述

A Survey of Deep Learning Library Testing Methods

Decentralized learning (DL) systems have been gaining popularity because they
avoid raw data sharing by communicating only model parameters, hence preserving
data confidentiality. However, the large size of deep neural networks poses a
significant challenge for decentralized training, since each node needs to
exchange gigabytes of data, overloading the network. In this paper, we address
this challenge with JWINS, a communication-efficient and fully decentralized
learning system that shares only a subset of parameters through sparsification.
JWINS uses wavelet transform to limit the information loss due to
sparsification and a randomized communication cut-off that reduces
communication usage without damaging the performance of trained models. We
demonstrate empirically with 96 DL nodes on non-IID datasets that JWINS can
achieve similar accuracies to full-sharing DL while sending up to 64% fewer
bytes. Additionally, on low communication budgets, JWINS outperforms the
state-of-the-art communication-efficient DL algorithm CHOCO-SGD by up to 4x in
terms of network savings and time.

本文介绍了 JWINS，一种通信高效且完全分散的学习系统，通过稀疏化仅共享少量参数，使用小波变换和随机通信切断降低信息损失并减少传输字节数，可在非 I ID 数据集上训练出接近于全共享的 DL 准确率，且可节省多达 64% 的字节数，在低通信预算下优于 CHOCO-SGD 算法。

去中心化学习系统中以较少为更多

Get More for Less in Decentralized Learning Systems

Graph Neural Networks (GNNs) are an emerging research field. This specialized
Deep Neural Network (DNN) architecture is capable of processing graph
structured data and bridges the gap between graph processing and Deep Learning
(DL). As graphs are everywhere, GNNs can be applied to various domains
including recommendation systems, computer vision, natural language processing,
biology and chemistry. With the rapid growing size of real world graphs, the
need for efficient and scalable GNN training solutions has come. Consequently,
many works proposing GNN systems have emerged throughout the past few years.
However, there is an acute lack of overview, categorization and comparison of
such systems. We aim to fill this gap by summarizing and categorizing important
methods and techniques for large-scale GNN solutions. In addition, we establish
connections between GNN systems, graph processing systems and DL systems.

本文总结和分类大规模图神经网络解决方案的重要方法和技术，并建立了图神经网络系统、图处理系统和深度学习系统之间的联系。

图神经网络分布式系统的演化及其在图处理和深度学习中的起源：一项调查

The Evolution of Distributed Systems for Graph Neural Networks and their  Origin in Graph Processing and Deep Learning: A Survey

Deep learning (DL) defines a new data-driven programming paradigm where the
internal system logic is largely shaped by the training data. The standard way
of evaluating DL models is to examine their performance on a test dataset. The
quality of the test dataset is of great importance to gain confidence of the
trained models. Using an inadequate test dataset, DL models that have achieved
high test accuracy may still lack generality and robustness. In traditional
software testing, mutation testing is a well-established technique for quality
evaluation of test suites, which analyzes to what extent a test suite detects
the injected faults. However, due to the fundamental difference between
traditional software and deep learning-based software, traditional mutation
testing techniques cannot be directly applied to DL systems. In this paper, we
propose a mutation testing framework specialized for DL systems to measure the
quality of test data. To do this, by sharing the same spirit of mutation
testing in traditional software, we first define a set of source-level mutation
operators to inject faults to the source of DL (i.e., training data and
training programs). Then we design a set of model-level mutation operators that
directly inject faults into DL models without a training process. Eventually,
the quality of test data could be evaluated from the analysis on to what extent
the injected faults could be detected. The usefulness of the proposed mutation
testing techniques is demonstrated on two public datasets, namely MNIST and
CIFAR-10, with three DL models.

本研究提出了一种专为深度学习系统设计的突变测试框架，通过注入故障来度量测试数据的质量，并在 MNIST 和 CIFAR-10 数据集上验证了其有效性。