The increasing compute demands of AI systems has led to the emergence of
services that train models on behalf of clients lacking necessary resources.
However, ensuring correctness of training and guarding against potential
training-time attacks, such as data poisoning, poses challenges. Existing works
on verifiable training largely fall into two classes: proof-based systems,
which struggle to scale due to requiring cryptographic techniques, and
"optimistic" methods that consider a trusted third-party auditor who replicates
the training process. A key challenge with the latter is that hardware
nondeterminism between GPU types during training prevents an auditor from
replicating the training process exactly, and such schemes are therefore
non-robust. We propose a method that combines training in a higher precision
than the target model, rounding after intermediate computation steps, and
storing rounding decisions based on an adaptive thresholding procedure, to
successfully control for nondeterminism. Across three different NVIDIA GPUs
(A40, Titan XP, RTX 2080 Ti), we achieve exact training replication at FP32
precision for both full-training and fine-tuning of ResNet-50 (23M) and GPT-2
(117M) models. Our verifiable training scheme significantly decreases the
storage and time costs compared to proof-based systems.

提出一种可验证训练的方法，通过在更高精度的模型上进行训练，舍入中间计算步骤后存储舍入决策，并基于自适应阈值过程，成功控制不确定性，从而实现了对训练过程的准确复制，降低了存储和时间成本。

通过控制硬件的非确定性实现乐观可验证训练

Optimistic Verifiable Training by Controlling Hardware Nondeterminism

Nondeterminism in neural network optimization produces uncertainty in
performance, making small improvements difficult to discern from run-to-run
variability. While uncertainty can be reduced by training multiple model
copies, doing so is time-consuming, costly, and harms reproducibility. In this
work, we establish an experimental protocol for understanding the effect of
optimization nondeterminism on model diversity, allowing us to isolate the
effects of a variety of sources of nondeterminism. Surprisingly, we find that
all sources of nondeterminism have similar effects on measures of model
diversity. To explain this intriguing fact, we identify the instability of
model training, taken as an end-to-end procedure, as the key determinant. We
show that even one-bit changes in initial parameters result in models
converging to vastly different values. Last, we propose two approaches for
reducing the effects of instability on run-to-run variability.

本文实验性地研究了神经网络优化的不确定性，证明了所有不确定性来源都对模型多样性具有类似的影响，这是因为模型训练的不稳定性是决定性的因素，并提出了两种减少波动性的方法。

神经网络优化中的不确定性和不稳定性

Nondeterminism and Instability in Neural Network Optimization

While deep reinforcement learning (DRL) has led to numerous successes in
recent years, reproducing these successes can be extremely challenging. One
reproducibility challenge particularly relevant to DRL is nondeterminism in the
training process, which can substantially affect the results. Motivated by this
challenge, we study the positive impacts of deterministic implementations in
eliminating nondeterminism in training. To do so, we consider the particular
case of the deep Q-learning algorithm, for which we produce a deterministic
implementation by identifying and controlling all sources of nondeterminism in
the training process. One by one, we then allow individual sources of
nondeterminism to affect our otherwise deterministic implementation, and
measure the impact of each source on the variance in performance. We find that
individual sources of nondeterminism can substantially impact the performance
of agent, illustrating the benefits of deterministic implementations. In
addition, we also discuss the important role of deterministic implementations
in achieving exact replicability of results.

本文研究了深度强化学习中训练的不确定性问题，并通过确定性实现来控制其表现差异，实验结果表明确定性实现能有效提高智能体的性能表现，并且对于结果的精确复现也具有重要作用。